-- Logs begin at Mon 2022-08-29 18:13:37 UTC, end at Mon 2022-08-29 19:29:27 UTC. -- Aug 29 18:47:09.356821 dv-cinder-850830-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 29 18:47:09.365951 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 29 18:47:09.367977 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 29 18:47:09 2022] *** Aug 29 18:47:09.367977 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 29 18:47:09.367977 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 29 18:47:09.367977 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: nodename: dv-cinder-850830-2-iscsi Aug 29 18:47:09.367977 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: machine: x86_64 Aug 29 18:47:09.367977 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: clock source: unix Aug 29 18:47:09.367977 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: pcre jit disabled Aug 29 18:47:09.367977 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: detected number of CPU cores: 4 Aug 29 18:47:09.367977 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: current working directory: / Aug 29 18:47:09.367977 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: detected binary path: /usr/bin/uwsgi-core Aug 29 18:47:09.368520 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: your processes number limit is 47680 Aug 29 18:47:09.368520 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: your memory page size is 4096 bytes Aug 29 18:47:09.368520 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: detected max file descriptor number: 2048 Aug 29 18:47:09.368520 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: lock engine: pthread robust mutexes Aug 29 18:47:09.368520 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: thunder lock: enabled Aug 29 18:47:09.368520 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 29 18:47:09.368520 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 29 18:47:09.407372 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: Python main interpreter initialized at 0x563fa611f9c0 Aug 29 18:47:09.407372 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: python threads support enabled Aug 29 18:47:09.407372 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: your server socket listen backlog is limited to 100 connections Aug 29 18:47:09.407372 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: your mercy for graceful operations on workers is 90 seconds Aug 29 18:47:09.407758 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: mapped 403077 bytes (393 KB) for 2 cores Aug 29 18:47:09.407837 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: *** Operational MODE: preforking *** Aug 29 18:47:09.408059 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: *** uWSGI is running in multiple interpreter mode *** Aug 29 18:47:09.408059 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: spawned uWSGI master process (pid: 81482) Aug 29 18:47:09.408268 dv-cinder-850830-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 29 18:47:09.410900 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: spawned uWSGI worker 1 (pid: 81499, cores: 1) Aug 29 18:47:09.410900 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: spawned uWSGI worker 2 (pid: 81500, cores: 1) Aug 29 18:47:09.410900 dv-cinder-850830-2-iscsi devstack@keystone.service[81482]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 29 18:47:13.711320 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:47:13.711320 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: from cryptography.utils import int_from_bytes Aug 29 18:47:13.712035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:47:13.712035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: from cryptography.utils import int_from_bytes Aug 29 18:47:13.713203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:47:13.713203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: from cryptography.utils import int_from_bytes Aug 29 18:47:13.713633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:47:13.713633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: from cryptography.utils import int_from_bytes Aug 29 18:47:13.761506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 18:47:13.761681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 18:47:13.761872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] command line args: [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 18:47:13.762142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 18:47:13.762317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ================================================================================ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 18:47:13.762572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] admin_token = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.762818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config_dir = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.763005 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 18:47:13.763097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.763201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config_source = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.763277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 18:47:13.763380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] debug = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.763464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] command line args: [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 18:47:13.763578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.763819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 18:47:13.763819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ================================================================================ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 18:47:13.763942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.764065 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] admin_token = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.764145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.764279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config_dir = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.764362 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.764458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.764569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.764662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config_source = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.764797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.764906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] debug = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.765001 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_config_append = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.765102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.765226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.765319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.765423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_dir = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.765526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.765630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.765732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.765839 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.765964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.766037 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.766154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.766225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.766305 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_config_append = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.766380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.766502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.766576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.766697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_dir = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.766778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.767389 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.767464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.767580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.767695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.767771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.767931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.768053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.768163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.768240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.768361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.768441 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.768522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.768637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.768736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.768847 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.768978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.769092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.769209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.769368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.769581 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.769813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.769885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.86/identity {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.770028 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] publish_errors = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.770094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.770261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.770333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.770493 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.770576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.770671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.770747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.770914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.771170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.771271 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.86/identity {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.771404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] standard_threads = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.771487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] publish_errors = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.771634 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.771722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.771875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.771946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.772047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.772120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.772218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_journal = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.772300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.772391 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_json = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.772508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.772590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_stderr = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.772725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_syslog = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.772846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] standard_threads = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.772953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.773034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.773189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.773264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.773362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.773445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.773520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_journal = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.773672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.773753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_json = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.773849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_stderr = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.773924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.774022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_syslog = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.774144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.774246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:47:13.774387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.774497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.774633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.774718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.774869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.774973 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.775137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.775232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.775572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.775667 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.775744 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.775820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.775889 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.775957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.776074 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.776161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.776309 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.776391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.776553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.776628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.776798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.776885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.777035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.777112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.777288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.777363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.777516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.777602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.777759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.777841 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.777990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.778065 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.778221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.778370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.778470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.778546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.778717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.778799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.778955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.779051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.779211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.779287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.779444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.779525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.779675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.779757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.779909 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.779984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.780157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.780232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.780384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.780464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.780612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.780687 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.780838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.780911 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.781067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.781141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.781310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.781385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.781544 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.781618 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.781769 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.781845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.781996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.782072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.782232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.782307 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.782485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.782593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.783285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.783372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.783521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.783598 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.783750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.783826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.783978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.784052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.784205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.784279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.784420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.784491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.784648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.784724 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.784876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.784953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.785104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.785180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.785338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.785414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.785574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.785649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.785799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.785874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.786032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.786106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.786260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.786341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.786495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.786567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.786724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.786789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.786958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.787041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.787207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.787273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.787437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.787539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.787669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.787745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.787903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.787976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.788135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.788211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.788365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.788448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.788589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.788664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.788833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.788908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.789061 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.789143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.789292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.789374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.789519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.789596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.789774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.789850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.790006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.790091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.790232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.790318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.790465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.790541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.790699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.790783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.790932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.791023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.791201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.791278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.791442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.791526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.791679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.791754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.791909 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.791985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.792142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.792225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.792374 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.792452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.792619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.792690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.792851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.792933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.793160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.793320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.793398 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.793553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.793635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.793789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.793866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.794021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.794095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.794340 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.794487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.794563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.794718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.794794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.794950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.795122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.795225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.795301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.795464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.795545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.795698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.795850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.795923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.795991 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.796152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.796227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.796382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.796467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.796612 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.796689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.796843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.796917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.797070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.797144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.797295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.797370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.797522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.797603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.797749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.797825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.797980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.798058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.798233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.798309 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.798472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.798548 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.798706 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.798783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.798950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.799055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.external = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.799209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.799293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.799460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.799546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.799716 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.799794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.799946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.password = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.800021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.800180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.token = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.800258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.external = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.800414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.800491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.800656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.800882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.800974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.801114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.801202 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.801345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.password = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.801422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.801573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.token = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.801649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.801808 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.801895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.802040 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.802121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.802282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.802359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.802511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.802600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.802754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.802831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.803035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.803115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.803248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.803334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.803479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.803562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.803721 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.803804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.803963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.804048 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.804199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.804275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.804437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.804512 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.804681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.804766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.804913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.804997 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.805161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.805238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.805409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.805499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.805643 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.805721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.805889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.805965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.806129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.806212 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.806374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.806453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.806619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.806696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.806849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.806934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.807118 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.807197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.807354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.807430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.807590 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.807669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.807828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.807904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.808069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.808151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.808296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.808375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.808538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.808615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.808771 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.808846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.809016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.809093 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.809249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.809323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.809481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.809557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.809718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.809798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.809959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.810034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.810189 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.810264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.810429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.810507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.810668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.810743 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.810900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.810977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.811153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.811240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.811383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.811459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.811613 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.811689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.811849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.811928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.812092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.812169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.812338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.812414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.812565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.812641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.812792 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.812869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.813018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.813095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.813259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.813337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.813500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.813577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.813768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.813845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.814008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.814085 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.814240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.814316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.814482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.814565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.814723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.814799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.814959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.815063 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.815210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.815286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.815444 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.815523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.815672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.815749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.815905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.815981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.816140 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.816267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.816387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.816509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.816631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.816754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.816863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.816980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.817101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.817222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.817331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.817466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.817569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.817698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.817806 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.817939 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.818046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.818181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.818285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.818406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.818522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.818641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.818769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.818853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.819311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.819398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.819547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.819623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.819776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.819865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.820026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.820103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.820272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.820348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.820507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.820592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.820742 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.820819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.820969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.821043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.821197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.821280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.821435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.821518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.821675 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.821751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.821905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.821994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.822137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.822214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.822369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.822445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.822598 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.822831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.822906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.823090 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.823170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.823318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.823404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.823550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.823625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.823786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.823862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.824016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.824100 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.824244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.824319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.824470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.824546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.824700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.824784 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.824933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.825009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.825164 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.825241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.825390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.825474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.825632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.825708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.825866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.825941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.826102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.826189 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.826337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.826413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.826574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.826656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.826796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.826879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.827057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.827139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.827294 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.827371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.827527 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.827616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.827762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.827839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.827994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.828070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.828229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.828312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.828465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.828542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.828697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.828772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.828946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.829030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.829175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.829275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.829940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.830019 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.830177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.830263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.830409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.830487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.830642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.830719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.830961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.831063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.831222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.831300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.831471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.831548 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.831721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.831805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.831959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.832121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.832197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.832373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.832450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.832620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.832704 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.832856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.832933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.833102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.833251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.833324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.833392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.833571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.833648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.833812 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.833895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.834048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.834125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.834284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.834361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.834516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.834601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.834760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.834847 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.835006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.835085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.835261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.835349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.835494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.835571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.835738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.835814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.835980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.836063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.836220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.836296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.836463 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.836541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.836707 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.836858 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.836936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.837095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.837171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.837343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.837420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.837592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.837670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.837839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.837917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.838080 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.838155 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.838312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.838465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.838564 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.838697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.838817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.838943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.839086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.839212 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.839290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.839457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.839534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.839700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.839779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.839930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.840008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.840181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.840258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.840431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.840509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.840666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.840743 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.840895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.840973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.841123 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.841200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.841348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.841424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.841583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.841658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.841829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.841907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.842065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.842142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.842294 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.842369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.842526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.842604 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.842751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.842828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.842982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.843076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.843235 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.843313 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.843476 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.843553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.843705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.843782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.843949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.844026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.844185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.844261 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.844413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.844495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.844653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.844731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.844898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.845597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.845716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.845827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.845985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.846071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.846247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.846324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.846491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.846567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.846737 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.846822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.846976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.847110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.847238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.847316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.847473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.847560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.847710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.847795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.847946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.848023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.848182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.848258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.848424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.848508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.848656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.848732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.848892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.848969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.849122 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.849207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.849356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.849435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.849587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.849663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.849831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.849916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.850060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.850137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.850297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.850374 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.850532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.850617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.850689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 18:47:13.850920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.851190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.851424 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.851665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.851893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.852128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.852359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:47:13.852524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 18:47:13.880600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.880835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.881030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.881316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.881725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.881997 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.889473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.890384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.918375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.918603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.919205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.919433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.972315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:13.972794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.021412 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.021647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.021736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.021967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.177897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.178119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.178209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.178340 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.178451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.178557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.178641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.178772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.178854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.178986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.179088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.179250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.202756 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.202990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.203279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.203383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.203506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.203587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.203719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.203799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.203931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.204014 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.204140 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.204222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.204406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.205267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.215952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.216058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.229213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.229396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.229537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.229631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.229708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.229822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.229905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.230034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.230125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.230244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.230326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.230453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.230534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.230701 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:47:14.247688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.247891 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:47:14.248005 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.248215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:47:14.248296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.248417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:47:14.248553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.248690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.248772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:47:14.248953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.249041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.249212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.249292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.249469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.249548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.249722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.259010 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.259301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.259383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.259572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.259654 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.259853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.259937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.260129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.266773 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.267076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.269765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.270047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.270136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.270326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.270409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.270605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.270700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.270885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.270967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.271172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.271255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.271439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.271521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.271702 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.271784 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.271972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.272060 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.272232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.272311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.272499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.272579 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.272762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.295970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.296232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.297422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.297703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.297785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.297990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.302896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.303194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.303278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.303473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.303569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.303757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.309861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.310145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.310225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.310474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.314094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.314383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.314480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.314663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.314743 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.314941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.315042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.315234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.315315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.315544 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.317215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.317498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.317578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.317777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.317857 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.318102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.333922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.334194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.334301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.334585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.338129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.338390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.341175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.341459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.341539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.341758 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.346257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.346592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.349373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.349678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.352549 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.352749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.352835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.353051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.353132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.353337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.353427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.353624 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.353704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.353907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.353988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.354290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.370119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.370403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.370493 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.370667 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.370753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.370928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.371035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.371295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.378168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.378541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.379856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.380220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.381513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.381760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.381851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.382038 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.384414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.384678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.384760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.384963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.385054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.385252 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.385334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.385539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.385631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.385829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.396850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.397135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.397247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.397412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.397531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.397688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.397815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.397966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.398099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.398237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.398380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.398524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.398656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.398952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.413737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.414243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.416763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.417310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.419651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.420212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.421068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.421652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.422525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.422812 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.423650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.423740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.423951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.424254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.430265 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.430685 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.431759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.432190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.434588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.434883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.435094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.435181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.435398 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.435494 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.435688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.435965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.444248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.444538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.444679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.444815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.444971 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.445251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.451680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.451962 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.452153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:47:14.452237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.452436 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.452518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.452717 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.452809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.452993 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.453074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.453259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.453339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.453529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.453621 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.453803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.453883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.454071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.454152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.454362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.454443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.454621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.454747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.454882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.454963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.455170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.455252 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.455437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.455714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:47:14.489587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563fa611f9c0 pid: 81500 (default app) Aug 29 18:47:14.489703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563fa611f9c0 pid: 81499 (default app) Aug 29 18:47:14.493055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dc9a-a7f2-47a1-9972-2fcce6ca381a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:14.493342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dc9a-a7f2-47a1-9972-2fcce6ca381a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:14.493611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dc9a-a7f2-47a1-9972-2fcce6ca381a None None] PATH_INFO: `/v3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:14.498360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1/1] 10.222.0.86 () {50 vars in 791 bytes} [Mon Aug 29 18:47:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:21.112550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877bdf1-b422-4187-944f-74449019c469 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:21.113126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877bdf1-b422-4187-944f-74449019c469 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:21.113126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877bdf1-b422-4187-944f-74449019c469 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:21.117540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1/2] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:47:21.122710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590624c-7daa-4b07-8b4e-763f9c8792d9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:21.123253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590624c-7daa-4b07-8b4e-763f9c8792d9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:21.123350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590624c-7daa-4b07-8b4e-763f9c8792d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:21.193757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: INFO dbcounter [None req-0590624c-7daa-4b07-8b4e-763f9c8792d9 None None] Registered counter for database keystone Aug 29 18:47:21.212953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG oslo_db.sqlalchemy.engines [None req-0590624c-7daa-4b07-8b4e-763f9c8792d9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81500) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 18:47:21.219192 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writer thread running {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 18:47:21.406204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG passlib.handlers.bcrypt [None req-0590624c-7daa-4b07-8b4e-763f9c8792d9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81500) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 18:47:21.406501 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG passlib.handlers.bcrypt [None req-0590624c-7daa-4b07-8b4e-763f9c8792d9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81500) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 18:47:21.464700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0590624c-7daa-4b07-8b4e-763f9c8792d9 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:21.466289 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0590624c-7daa-4b07-8b4e-763f9c8792d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:21.593888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 2/3] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 18:47:22.028440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877bdf1-b422-4187-944f-74449019c469 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:22.098542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: INFO dbcounter [None req-b877bdf1-b422-4187-944f-74449019c469 None None] Registered counter for database keystone Aug 29 18:47:22.119430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG oslo_db.sqlalchemy.engines [None req-b877bdf1-b422-4187-944f-74449019c469 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81499) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 18:47:22.125572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writer thread running {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 18:47:22.246243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877bdf1-b422-4187-944f-74449019c469 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:22.250522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe7c33-ba32-4665-876f-7a4e8e786362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:22.251697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe7c33-ba32-4665-876f-7a4e8e786362 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:22.251981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe7c33-ba32-4665-876f-7a4e8e786362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:22.252254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe7c33-ba32-4665-876f-7a4e8e786362 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:22.260412 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe7c33-ba32-4665-876f-7a4e8e786362 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:22.311768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: WARNING keystone.common.rbac_enforcer.enforcer [None req-90fe7c33-ba32-4665-876f-7a4e8e786362 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 18:47:22.312058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe7c33-ba32-4665-876f-7a4e8e786362 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:22.322406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 2/4] 10.222.0.86 () {58 vars in 1324 bytes} [Mon Aug 29 18:47:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 18:47:22.328356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590624c-7daa-4b07-8b4e-763f9c8792d9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:22.342933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590624c-7daa-4b07-8b4e-763f9c8792d9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:22.347282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82134dd1-eee8-4294-9b72-27b1a481e18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:22.348367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82134dd1-eee8-4294-9b72-27b1a481e18b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:22.348646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82134dd1-eee8-4294-9b72-27b1a481e18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:22.348917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82134dd1-eee8-4294-9b72-27b1a481e18b admin admin] PATH_INFO: `/v3/projects/3e9f996197414d24b783bd100f720103` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:22.350455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82134dd1-eee8-4294-9b72-27b1a481e18b admin admin] RBAC: Authorizing `identity:get_project(project_id=3e9f996197414d24b783bd100f720103)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:22.403111 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: WARNING keystone.common.rbac_enforcer.enforcer [None req-82134dd1-eee8-4294-9b72-27b1a481e18b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 18:47:22.403451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82134dd1-eee8-4294-9b72-27b1a481e18b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:22.408037 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 3/5] 10.222.0.86 () {56 vars in 1207 bytes} [Mon Aug 29 18:47:22 2022] GET /identity/v3/projects/3e9f996197414d24b783bd100f720103 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:24.296115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c8cc-79bc-4d63-aba6-b87db983ea7a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:24.296619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c8cc-79bc-4d63-aba6-b87db983ea7a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:24.296698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c8cc-79bc-4d63-aba6-b87db983ea7a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:24.300970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 3/6] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:47:24.306419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93319-312a-4ece-9b55-529b0c2cc365 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:24.306801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93319-312a-4ece-9b55-529b0c2cc365 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:24.306968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93319-312a-4ece-9b55-529b0c2cc365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:24.326784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-73a93319-312a-4ece-9b55-529b0c2cc365 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:24.328304 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-73a93319-312a-4ece-9b55-529b0c2cc365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:24.336479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 4/7] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 29 18:47:24.602465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1c8cc-79bc-4d63-aba6-b87db983ea7a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:24.615706 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1c8cc-79bc-4d63-aba6-b87db983ea7a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:24.619884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19cb06-69ba-4446-b936-3a221613b276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:24.620478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19cb06-69ba-4446-b936-3a221613b276 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:24.620756 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19cb06-69ba-4446-b936-3a221613b276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:24.621025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19cb06-69ba-4446-b936-3a221613b276 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:24.629269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19cb06-69ba-4446-b936-3a221613b276 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:24.631054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19cb06-69ba-4446-b936-3a221613b276 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:24.639684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 4/8] 10.222.0.86 () {58 vars in 1324 bytes} [Mon Aug 29 18:47:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 18:47:24.645583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a93319-312a-4ece-9b55-529b0c2cc365 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:24.654548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a93319-312a-4ece-9b55-529b0c2cc365 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:24.658848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90e005-002e-4e1f-8578-c6cfb4acac42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:24.659510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e005-002e-4e1f-8578-c6cfb4acac42 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:24.659789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e005-002e-4e1f-8578-c6cfb4acac42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:24.660061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e005-002e-4e1f-8578-c6cfb4acac42 admin admin] PATH_INFO: `/v3/users/9c8f433f2f1f4923859810dc0019845c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:24.661370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90e005-002e-4e1f-8578-c6cfb4acac42 admin admin] RBAC: Authorizing `identity:get_user(user_id=9c8f433f2f1f4923859810dc0019845c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:24.663215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90e005-002e-4e1f-8578-c6cfb4acac42 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:24.664740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 5/9] 10.222.0.86 () {56 vars in 1198 bytes} [Mon Aug 29 18:47:24 2022] GET /identity/v3/users/9c8f433f2f1f4923859810dc0019845c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:26.621809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b691a3b-2b84-4591-bbfc-a561612a5f41 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:26.622373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b691a3b-2b84-4591-bbfc-a561612a5f41 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:26.622373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b691a3b-2b84-4591-bbfc-a561612a5f41 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:26.623691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 5/10] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:26.628696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d02b20-82b7-4614-85eb-fcc023f70b63 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:26.629104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d02b20-82b7-4614-85eb-fcc023f70b63 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:26.629367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d02b20-82b7-4614-85eb-fcc023f70b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:26.645900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd0086-f85b-4f48-98d4-2d907cd7fb0e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:26.646179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd0086-f85b-4f48-98d4-2d907cd7fb0e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:26.646452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd0086-f85b-4f48-98d4-2d907cd7fb0e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:26.647772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 6/11] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:47:26.647971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-77d02b20-82b7-4614-85eb-fcc023f70b63 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:26.649150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-77d02b20-82b7-4614-85eb-fcc023f70b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:26.652700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c10c7-c11e-4815-a1ca-92a7122c1eb2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:26.652981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c10c7-c11e-4815-a1ca-92a7122c1eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:26.653255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c10c7-c11e-4815-a1ca-92a7122c1eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:26.695573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG passlib.handlers.bcrypt [None req-b46c10c7-c11e-4815-a1ca-92a7122c1eb2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81499) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 18:47:26.695966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG passlib.handlers.bcrypt [None req-b46c10c7-c11e-4815-a1ca-92a7122c1eb2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81499) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 18:47:26.729488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 6/12] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:26.746385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b46c10c7-c11e-4815-a1ca-92a7122c1eb2 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:26.748018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b46c10c7-c11e-4815-a1ca-92a7122c1eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:26.847783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 7/13] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:26.995331 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d02b20-82b7-4614-85eb-fcc023f70b63 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:27.019912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:27.099522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d02b20-82b7-4614-85eb-fcc023f70b63 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:27.111488 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c10c7-c11e-4815-a1ca-92a7122c1eb2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:27.133711 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce41f29-1d59-43aa-9250-61ed6deec4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:27.134355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce41f29-1d59-43aa-9250-61ed6deec4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:27.134633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce41f29-1d59-43aa-9250-61ed6deec4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:27.134906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce41f29-1d59-43aa-9250-61ed6deec4f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:27.145708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce41f29-1d59-43aa-9250-61ed6deec4f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2ce41f29-1d59-43aa-9250-61ed6deec4f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:27.154554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:27.155550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:47:27.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 7/14] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:47:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:27.161395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce41f29-1d59-43aa-9250-61ed6deec4f4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:27.211895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c10c7-c11e-4815-a1ca-92a7122c1eb2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:27.214163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:27.241916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:27.250583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf910da9-a8a3-47e7-a6bf-5b31b91051df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:27.251919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf910da9-a8a3-47e7-a6bf-5b31b91051df None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:27.252199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf910da9-a8a3-47e7-a6bf-5b31b91051df None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:27.252467 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf910da9-a8a3-47e7-a6bf-5b31b91051df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:27.260600 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce41f29-1d59-43aa-9250-61ed6deec4f4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:27.295515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00038b5-d121-4a5f-a23a-5998f143dfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:27.296119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00038b5-d121-4a5f-a23a-5998f143dfa9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:27.296402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00038b5-d121-4a5f-a23a-5998f143dfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:27.296672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00038b5-d121-4a5f-a23a-5998f143dfa9 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:27.297829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00038b5-d121-4a5f-a23a-5998f143dfa9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:27.298140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00038b5-d121-4a5f-a23a-5998f143dfa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:27.299547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00038b5-d121-4a5f-a23a-5998f143dfa9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:27.311729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 8/15] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:27.318742 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf910da9-a8a3-47e7-a6bf-5b31b91051df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:27.320167 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00038b5-d121-4a5f-a23a-5998f143dfa9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:27.320757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf910da9-a8a3-47e7-a6bf-5b31b91051df None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:27.416088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00038b5-d121-4a5f-a23a-5998f143dfa9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:27.419193 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 8/16] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:47:27.421651 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf910da9-a8a3-47e7-a6bf-5b31b91051df None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:27.424931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:27.451174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca7caa-cf79-43b7-a8c1-40e3c36fb4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:27.451812 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca7caa-cf79-43b7-a8c1-40e3c36fb4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:27.452097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca7caa-cf79-43b7-a8c1-40e3c36fb4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:27.452368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca7caa-cf79-43b7-a8c1-40e3c36fb4b9 None admin] PATH_INFO: `/v3/users/9c8f433f2f1f4923859810dc0019845c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:27.453497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca7caa-cf79-43b7-a8c1-40e3c36fb4b9 None admin] RBAC: Authorizing `identity:get_user(user_id=9c8f433f2f1f4923859810dc0019845c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:27.454976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca7caa-cf79-43b7-a8c1-40e3c36fb4b9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:27.456572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 9/17] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:27 2022] GET /identity/v3/users/9c8f433f2f1f4923859810dc0019845c => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:27.462035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca7caa-cf79-43b7-a8c1-40e3c36fb4b9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:27.476369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:27.519933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf910da9-a8a3-47e7-a6bf-5b31b91051df None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:27.554083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc1d02-4a87-442a-8d18-d05863537f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:27.554939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc1d02-4a87-442a-8d18-d05863537f6a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:27.555250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc1d02-4a87-442a-8d18-d05863537f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:27.555526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc1d02-4a87-442a-8d18-d05863537f6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:27.560600 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca7caa-cf79-43b7-a8c1-40e3c36fb4b9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:27.573711 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc1d02-4a87-442a-8d18-d05863537f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:27.575212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc1d02-4a87-442a-8d18-d05863537f6a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-58fc1d02-4a87-442a-8d18-d05863537f6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:47:27.593077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:27.593974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:47:27.595328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:47:27.595328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:47:27.595328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:27.595328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 9/18] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:47:27.595545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667245b-f5aa-42ee-ad43-cb3f11b9d8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:27.596098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667245b-f5aa-42ee-ad43-cb3f11b9d8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:27.596377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667245b-f5aa-42ee-ad43-cb3f11b9d8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:27.596658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667245b-f5aa-42ee-ad43-cb3f11b9d8b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:27.597780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667245b-f5aa-42ee-ad43-cb3f11b9d8b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:27.599246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667245b-f5aa-42ee-ad43-cb3f11b9d8b1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:27.600425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc1d02-4a87-442a-8d18-d05863537f6a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:27.600982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 10/19] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:27.606239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667245b-f5aa-42ee-ad43-cb3f11b9d8b1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:27.659331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:27.690229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:27.704012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc1d02-4a87-442a-8d18-d05863537f6a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:27.709404 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667245b-f5aa-42ee-ad43-cb3f11b9d8b1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:27.737093 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd961d1-ec3d-4083-97c0-87a36dcfa73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:27.737810 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd961d1-ec3d-4083-97c0-87a36dcfa73b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:27.738153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd961d1-ec3d-4083-97c0-87a36dcfa73b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:27.738496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd961d1-ec3d-4083-97c0-87a36dcfa73b None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:27.739169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd961d1-ec3d-4083-97c0-87a36dcfa73b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:27.739707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd961d1-ec3d-4083-97c0-87a36dcfa73b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:27.740850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd961d1-ec3d-4083-97c0-87a36dcfa73b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:27.744069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77306dfc-572a-4f37-b219-e842d2ebc53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:27.744667 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77306dfc-572a-4f37-b219-e842d2ebc53a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:27.744944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77306dfc-572a-4f37-b219-e842d2ebc53a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:27.745218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77306dfc-572a-4f37-b219-e842d2ebc53a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:27.745931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77306dfc-572a-4f37-b219-e842d2ebc53a None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.domain.id=default, user.id=9c8f433f2f1f4923859810dc0019845c) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:27.746239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77306dfc-572a-4f37-b219-e842d2ebc53a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:27.747687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77306dfc-572a-4f37-b219-e842d2ebc53a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:27.754305 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 11/20] 10.222.0.86 () {56 vars in 1322 bytes} [Mon Aug 29 18:47:27 2022] GET /identity/v3/role_assignments?user.id=9c8f433f2f1f4923859810dc0019845c&scope.domain.id=default&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:27.757916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 10/21] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:29.915234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3b5ae-bdad-4aac-af36-c13af2b4abec None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:29.916603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3b5ae-bdad-4aac-af36-c13af2b4abec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:29.917293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3b5ae-bdad-4aac-af36-c13af2b4abec None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:29.919770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 12/22] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:29.932821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a633d-e7a2-409e-a541-07613a86bd9e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:29.933734 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a633d-e7a2-409e-a541-07613a86bd9e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:29.934231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a633d-e7a2-409e-a541-07613a86bd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:29.964110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3c1a633d-e7a2-409e-a541-07613a86bd9e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:29.965643 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3c1a633d-e7a2-409e-a541-07613a86bd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:30.056632 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 11/23] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:30.074213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675ea12-c506-4126-af38-0fe938837b55 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:30.074838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675ea12-c506-4126-af38-0fe938837b55 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:30.075324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675ea12-c506-4126-af38-0fe938837b55 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:30.077135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 13/24] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:30.082536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd50810-a142-4164-bbf3-afb01444a6e8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:30.083074 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd50810-a142-4164-bbf3-afb01444a6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:30.083556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd50810-a142-4164-bbf3-afb01444a6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:30.105372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2fd50810-a142-4164-bbf3-afb01444a6e8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:30.106877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2fd50810-a142-4164-bbf3-afb01444a6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:30.152271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:30.180359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 12/25] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:30.456154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675ea12-c506-4126-af38-0fe938837b55 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:30.579765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6685-b474-48f0-bcae-fd25c4187842 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:30.580441 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6685-b474-48f0-bcae-fd25c4187842 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:30.580915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6685-b474-48f0-bcae-fd25c4187842 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:30.582724 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 13/26] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:30.587619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac7693-4c79-4cc6-b7b3-802bd8b402ec None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:30.588178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac7693-4c79-4cc6-b7b3-802bd8b402ec None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:30.588658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac7693-4c79-4cc6-b7b3-802bd8b402ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:30.599020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675ea12-c506-4126-af38-0fe938837b55 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:30.624059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-eeac7693-4c79-4cc6-b7b3-802bd8b402ec None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:30.626185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eeac7693-4c79-4cc6-b7b3-802bd8b402ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:30.643077 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c2178-2c50-4fd6-ac2e-545b479070d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:30.644049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c2178-2c50-4fd6-ac2e-545b479070d9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:30.644543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c2178-2c50-4fd6-ac2e-545b479070d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:30.645014 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c2178-2c50-4fd6-ac2e-545b479070d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:30.658264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c2178-2c50-4fd6-ac2e-545b479070d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a66c2178-2c50-4fd6-ac2e-545b479070d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:30.664075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:30.664932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:30.665693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:47:30.667970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 14/27] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:47:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:30.669399 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59331500-6dec-4376-989f-499cfbe77caf None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:30.669906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59331500-6dec-4376-989f-499cfbe77caf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:30.670371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59331500-6dec-4376-989f-499cfbe77caf None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:30.672312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 15/28] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:30.673283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59331500-6dec-4376-989f-499cfbe77caf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:30.685532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:30.735599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 14/29] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:30.738420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3d69b-65f3-4da0-9c02-f7a0b56d826a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:30.738708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3d69b-65f3-4da0-9c02-f7a0b56d826a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:30.740388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3d69b-65f3-4da0-9c02-f7a0b56d826a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:30.763240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-01f3d69b-65f3-4da0-9c02-f7a0b56d826a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:30.764689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-01f3d69b-65f3-4da0-9c02-f7a0b56d826a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:30.876731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59331500-6dec-4376-989f-499cfbe77caf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:30.905454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 15/30] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:30.906521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3d69b-65f3-4da0-9c02-f7a0b56d826a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:30.949494 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b3ab9-c264-4710-a7ce-f3c81a9ba775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:30.950738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b3ab9-c264-4710-a7ce-f3c81a9ba775 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:30.951187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b3ab9-c264-4710-a7ce-f3c81a9ba775 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:30.951620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b3ab9-c264-4710-a7ce-f3c81a9ba775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:30.973568 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b3ab9-c264-4710-a7ce-f3c81a9ba775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:30.975094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b3ab9-c264-4710-a7ce-f3c81a9ba775 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:30.975811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-cb8b3ab9-c264-4710-a7ce-f3c81a9ba775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:47:31.004546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:31.005555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:47:31.006380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:47:31.006380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:47:31.006380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:47:31.006380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 16/31] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:31.011752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b3ab9-c264-4710-a7ce-f3c81a9ba775 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:31.053934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3d69b-65f3-4da0-9c02-f7a0b56d826a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:31.070860 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:31.105063 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad8294-a3d0-4a9c-926d-6f681f280886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:31.106090 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8294-a3d0-4a9c-926d-6f681f280886 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:31.106593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8294-a3d0-4a9c-926d-6f681f280886 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:31.107127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8294-a3d0-4a9c-926d-6f681f280886 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:31.108035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad8294-a3d0-4a9c-926d-6f681f280886 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:31.108533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad8294-a3d0-4a9c-926d-6f681f280886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:31.110662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad8294-a3d0-4a9c-926d-6f681f280886 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:31.118281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b3ab9-c264-4710-a7ce-f3c81a9ba775 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:31.153287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 16/32] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:31.154207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad8294-a3d0-4a9c-926d-6f681f280886 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:31.154848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64875270-f510-4b9a-a15a-7774de4040d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:31.155607 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64875270-f510-4b9a-a15a-7774de4040d9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:31.155897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64875270-f510-4b9a-a15a-7774de4040d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:31.156173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64875270-f510-4b9a-a15a-7774de4040d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:31.156886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64875270-f510-4b9a-a15a-7774de4040d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:31.157194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64875270-f510-4b9a-a15a-7774de4040d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:31.158835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64875270-f510-4b9a-a15a-7774de4040d9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:31.181383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 17/33] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:31.182315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64875270-f510-4b9a-a15a-7774de4040d9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:31.308323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64875270-f510-4b9a-a15a-7774de4040d9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:31.327704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:31.330598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:31.335042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad8294-a3d0-4a9c-926d-6f681f280886 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:31.351211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5362b1-d2b1-46ea-9740-332e678c1a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:31.352183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5362b1-d2b1-46ea-9740-332e678c1a90 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:31.352603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5362b1-d2b1-46ea-9740-332e678c1a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:31.353050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5362b1-d2b1-46ea-9740-332e678c1a90 None admin] PATH_INFO: `/v3/users/9c8f433f2f1f4923859810dc0019845c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:31.354453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5362b1-d2b1-46ea-9740-332e678c1a90 None admin] RBAC: Authorizing `identity:get_user(user_id=9c8f433f2f1f4923859810dc0019845c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:31.356213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5362b1-d2b1-46ea-9740-332e678c1a90 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:31.358188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 18/34] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:31 2022] GET /identity/v3/users/9c8f433f2f1f4923859810dc0019845c => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:31.359089 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5362b1-d2b1-46ea-9740-332e678c1a90 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:31.379136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7c24f-729a-4d78-b178-a60a5b0c3f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:31.380192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7c24f-729a-4d78-b178-a60a5b0c3f20 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:31.380631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7c24f-729a-4d78-b178-a60a5b0c3f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:31.381006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7c24f-729a-4d78-b178-a60a5b0c3f20 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:31.382342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7c24f-729a-4d78-b178-a60a5b0c3f20 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:31.384277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7c24f-729a-4d78-b178-a60a5b0c3f20 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:31.476410 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5362b1-d2b1-46ea-9740-332e678c1a90 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:31.511874 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91801a4b-c6bf-4de9-bbde-0461ec127320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:31.512482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801a4b-c6bf-4de9-bbde-0461ec127320 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:31.512762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801a4b-c6bf-4de9-bbde-0461ec127320 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:31.513035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801a4b-c6bf-4de9-bbde-0461ec127320 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:31.513688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91801a4b-c6bf-4de9-bbde-0461ec127320 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:31.515143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91801a4b-c6bf-4de9-bbde-0461ec127320 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:31.531306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 17/35] 10.222.0.86 () {60 vars in 1131 bytes} [Mon Aug 29 18:47:31 2022] POST /identity/v3/roles => generated 225 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:47:31.531306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7c24f-729a-4d78-b178-a60a5b0c3f20 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:31.643425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7c24f-729a-4d78-b178-a60a5b0c3f20 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:31.688815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e622b5c-27fb-462a-aa15-850d199b20f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:31.689513 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622b5c-27fb-462a-aa15-850d199b20f1 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:31.689799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622b5c-27fb-462a-aa15-850d199b20f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:31.690076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622b5c-27fb-462a-aa15-850d199b20f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:31.690759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e622b5c-27fb-462a-aa15-850d199b20f1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:31.693700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e622b5c-27fb-462a-aa15-850d199b20f1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:31.696942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 19/36] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:47:31 2022] POST /identity/v3/projects => generated 301 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:47:31.703110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91801a4b-c6bf-4de9-bbde-0461ec127320 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:31.731795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 18/37] 10.222.0.86 () {60 vars in 1131 bytes} [Mon Aug 29 18:47:31 2022] POST /identity/v3/roles => generated 219 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:47:31.733464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c07aa3-7525-4dd5-9c50-f369ea9c5ca6 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:31.733881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c07aa3-7525-4dd5-9c50-f369ea9c5ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:31.734229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c07aa3-7525-4dd5-9c50-f369ea9c5ca6 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:31.736618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 19/38] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:31.742504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477d2bd-c45a-45ad-8be0-73f53e181636 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:31.742962 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477d2bd-c45a-45ad-8be0-73f53e181636 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:31.743359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477d2bd-c45a-45ad-8be0-73f53e181636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:31.771251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:31.787630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4477d2bd-c45a-45ad-8be0-73f53e181636 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:31.789368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4477d2bd-c45a-45ad-8be0-73f53e181636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:31.812212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:31.893089 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91801a4b-c6bf-4de9-bbde-0461ec127320 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:31.906054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 20/39] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:31.908364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f0d6-ff44-4ba6-b4b8-ef641ae268cd None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:31.908897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f0d6-ff44-4ba6-b4b8-ef641ae268cd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:31.909338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f0d6-ff44-4ba6-b4b8-ef641ae268cd None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:31.910975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 21/40] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:31.916491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255a568-689c-4171-bed4-dd6cccd9d884 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:31.916921 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255a568-689c-4171-bed4-dd6cccd9d884 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:31.917266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255a568-689c-4171-bed4-dd6cccd9d884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:31.939842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4255a568-689c-4171-bed4-dd6cccd9d884 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:31.941184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4255a568-689c-4171-bed4-dd6cccd9d884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:31.955738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaeae7d-5824-48a6-afbf-8859c355fd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:31.957184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeae7d-5824-48a6-afbf-8859c355fd7a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:31.957671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeae7d-5824-48a6-afbf-8859c355fd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:31.958301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeae7d-5824-48a6-afbf-8859c355fd7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:31.959975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaeae7d-5824-48a6-afbf-8859c355fd7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:31.962577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaeae7d-5824-48a6-afbf-8859c355fd7a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:31.965008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 20/41] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:31.971154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaeae7d-5824-48a6-afbf-8859c355fd7a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:32.033451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 22/42] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:32.082704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaeae7d-5824-48a6-afbf-8859c355fd7a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:32.085050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:32.195291 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255a568-689c-4171-bed4-dd6cccd9d884 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:32.226560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a74c4c-600d-4066-88c3-cb27cf8a0c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:32.227258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a74c4c-600d-4066-88c3-cb27cf8a0c02 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:32.227548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a74c4c-600d-4066-88c3-cb27cf8a0c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:32.227831 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a74c4c-600d-4066-88c3-cb27cf8a0c02 None admin] PATH_INFO: `/v3/domains/default/users/9c8f433f2f1f4923859810dc0019845c/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:32.229943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a74c4c-600d-4066-88c3-cb27cf8a0c02 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9c8f433f2f1f4923859810dc0019845c, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:32.231538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a74c4c-600d-4066-88c3-cb27cf8a0c02 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:32.247548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 21/43] 10.222.0.86 () {58 vars in 1361 bytes} [Mon Aug 29 18:47:31 2022] PUT /identity/v3/domains/default/users/9c8f433f2f1f4923859810dc0019845c/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:47:32.293224 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:32.302494 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a74c4c-600d-4066-88c3-cb27cf8a0c02 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:32.322945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255a568-689c-4171-bed4-dd6cccd9d884 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:32.363733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09ecb0-4614-4c0d-b5f6-c55836b6e337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:32.364523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09ecb0-4614-4c0d-b5f6-c55836b6e337 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:32.364810 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09ecb0-4614-4c0d-b5f6-c55836b6e337 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:32.365090 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09ecb0-4614-4c0d-b5f6-c55836b6e337 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:32.366530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09ecb0-4614-4c0d-b5f6-c55836b6e337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:32.368310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09ecb0-4614-4c0d-b5f6-c55836b6e337 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:32.374666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 23/44] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:32.375614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09ecb0-4614-4c0d-b5f6-c55836b6e337 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:32.424158 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a74c4c-600d-4066-88c3-cb27cf8a0c02 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:32.461917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa28b4-2158-41d4-9836-a53aec16af5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:32.462462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa28b4-2158-41d4-9836-a53aec16af5f None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:32.462751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa28b4-2158-41d4-9836-a53aec16af5f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:32.463057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa28b4-2158-41d4-9836-a53aec16af5f None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:32.463740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa28b4-2158-41d4-9836-a53aec16af5f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:32.465118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa28b4-2158-41d4-9836-a53aec16af5f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:32.482846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 22/45] 10.222.0.86 () {60 vars in 1131 bytes} [Mon Aug 29 18:47:32 2022] POST /identity/v3/roles => generated 223 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:47:32.494707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09ecb0-4614-4c0d-b5f6-c55836b6e337 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:32.533858 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2e501-3b7d-4726-a884-104ad21b04e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:32.534753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e501-3b7d-4726-a884-104ad21b04e7 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:32.535070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e501-3b7d-4726-a884-104ad21b04e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:32.535353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e501-3b7d-4726-a884-104ad21b04e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:32.536113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2e501-3b7d-4726-a884-104ad21b04e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:32.537626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2e501-3b7d-4726-a884-104ad21b04e7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:32.563177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:32.573823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 24/46] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:47:32 2022] POST /identity/v3/projects => generated 312 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:47:34.178075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e46b1e-43af-466f-abf3-d724ec805c0d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:34.178584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e46b1e-43af-466f-abf3-d724ec805c0d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:34.178688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e46b1e-43af-466f-abf3-d724ec805c0d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:34.179987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 23/47] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:34.185197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8c513-28ff-4d35-944a-60659583e052 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:34.185606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8c513-28ff-4d35-944a-60659583e052 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:34.185763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8c513-28ff-4d35-944a-60659583e052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:34.204494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-23b8c513-28ff-4d35-944a-60659583e052 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:34.205726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-23b8c513-28ff-4d35-944a-60659583e052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:34.271725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 25/48] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:34.543207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403eb722-c471-4565-aff4-22d0472b092a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:34.543581 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403eb722-c471-4565-aff4-22d0472b092a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:34.543897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403eb722-c471-4565-aff4-22d0472b092a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:34.545267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 24/49] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:34.549780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8c513-28ff-4d35-944a-60659583e052 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:34.550047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f551d2-59c5-47c9-8d27-79176fb9ca8f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:34.550339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f551d2-59c5-47c9-8d27-79176fb9ca8f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:34.550720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f551d2-59c5-47c9-8d27-79176fb9ca8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:34.569472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a4f551d2-59c5-47c9-8d27-79176fb9ca8f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:34.571016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a4f551d2-59c5-47c9-8d27-79176fb9ca8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:34.588723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:34.638810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 25/50] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:34.649908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8c513-28ff-4d35-944a-60659583e052 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:34.681034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:34.689683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cbe63-d95e-4517-820e-70ee85ce412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:34.690360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cbe63-d95e-4517-820e-70ee85ce412e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:34.691145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cbe63-d95e-4517-820e-70ee85ce412e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:34.691145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cbe63-d95e-4517-820e-70ee85ce412e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:34.705507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844cbe63-d95e-4517-820e-70ee85ce412e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-844cbe63-d95e-4517-820e-70ee85ce412e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:34.712751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:34.713595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:34.714343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 26/51] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:47:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:34.719726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f551d2-59c5-47c9-8d27-79176fb9ca8f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:34.817692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f551d2-59c5-47c9-8d27-79176fb9ca8f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:34.850472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7030f-0a9a-47cf-a8a1-4b824853ef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:34.851119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7030f-0a9a-47cf-a8a1-4b824853ef50 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:34.851406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7030f-0a9a-47cf-a8a1-4b824853ef50 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:34.851681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7030f-0a9a-47cf-a8a1-4b824853ef50 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:34.852342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7030f-0a9a-47cf-a8a1-4b824853ef50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:34.852647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7030f-0a9a-47cf-a8a1-4b824853ef50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:34.854032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7030f-0a9a-47cf-a8a1-4b824853ef50 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:34.862490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 26/52] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:34.868213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cbe63-d95e-4517-820e-70ee85ce412e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:34.910784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7030f-0a9a-47cf-a8a1-4b824853ef50 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:34.931818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:35.016797 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7030f-0a9a-47cf-a8a1-4b824853ef50 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:35.058821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e51571-b32c-4bbb-a077-5ce727ed08b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:35.059477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e51571-b32c-4bbb-a077-5ce727ed08b7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:35.059755 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e51571-b32c-4bbb-a077-5ce727ed08b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:35.060023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e51571-b32c-4bbb-a077-5ce727ed08b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:35.061138 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e51571-b32c-4bbb-a077-5ce727ed08b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:35.062585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e51571-b32c-4bbb-a077-5ce727ed08b7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:35.064369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 27/53] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:35.070924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e51571-b32c-4bbb-a077-5ce727ed08b7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:35.088500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cbe63-d95e-4517-820e-70ee85ce412e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:35.123490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bba0fe-fa13-4cd7-b131-983c54821bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:35.124117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bba0fe-fa13-4cd7-b131-983c54821bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:35.124396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bba0fe-fa13-4cd7-b131-983c54821bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:35.124668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bba0fe-fa13-4cd7-b131-983c54821bb0 None admin] PATH_INFO: `/v3/users/9c8f433f2f1f4923859810dc0019845c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:35.126033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bba0fe-fa13-4cd7-b131-983c54821bb0 None admin] RBAC: Authorizing `identity:get_user(user_id=9c8f433f2f1f4923859810dc0019845c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:35.127530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bba0fe-fa13-4cd7-b131-983c54821bb0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:35.129243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 27/54] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:34 2022] GET /identity/v3/users/9c8f433f2f1f4923859810dc0019845c => generated 247 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:35.134947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bba0fe-fa13-4cd7-b131-983c54821bb0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:35.157619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:35.174202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e51571-b32c-4bbb-a077-5ce727ed08b7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:35.194306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:35.208718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2fcfc-0043-4e3f-bb26-6c12a1f26a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:35.209328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2fcfc-0043-4e3f-bb26-6c12a1f26a43 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:35.209605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2fcfc-0043-4e3f-bb26-6c12a1f26a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:35.209877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2fcfc-0043-4e3f-bb26-6c12a1f26a43 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:35.210520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2fcfc-0043-4e3f-bb26-6c12a1f26a43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:35.211984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2fcfc-0043-4e3f-bb26-6c12a1f26a43 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:35.242610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bba0fe-fa13-4cd7-b131-983c54821bb0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:35.245514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 28/55] 10.222.0.86 () {60 vars in 1140 bytes} [Mon Aug 29 18:47:35 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:47:35.278125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2be85-e775-49a7-b7bb-34f35c677e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:35.278787 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2be85-e775-49a7-b7bb-34f35c677e8e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:35.279107 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2be85-e775-49a7-b7bb-34f35c677e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:35.279387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2be85-e775-49a7-b7bb-34f35c677e8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:35.280560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2be85-e775-49a7-b7bb-34f35c677e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:35.282530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2be85-e775-49a7-b7bb-34f35c677e8e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:35.284191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 28/56] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:35.290495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2fcfc-0043-4e3f-bb26-6c12a1f26a43 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:35.418150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2fcfc-0043-4e3f-bb26-6c12a1f26a43 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:35.455232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879ae62-6066-430f-b756-6cf1a8321b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:35.455916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879ae62-6066-430f-b756-6cf1a8321b55 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:35.456206 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879ae62-6066-430f-b756-6cf1a8321b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:35.456643 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879ae62-6066-430f-b756-6cf1a8321b55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:35.457377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879ae62-6066-430f-b756-6cf1a8321b55 None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.domain.id=default, user.id=9c8f433f2f1f4923859810dc0019845c) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:35.457688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879ae62-6066-430f-b756-6cf1a8321b55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:35.459155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879ae62-6066-430f-b756-6cf1a8321b55 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:35.462001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:35.465691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 29/57] 10.222.0.86 () {56 vars in 1322 bytes} [Mon Aug 29 18:47:35 2022] GET /identity/v3/role_assignments?user.id=9c8f433f2f1f4923859810dc0019845c&scope.domain.id=default&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 530 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:37.186276 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e7de7-3dd2-4ce5-b0b8-6f5bc4888260 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:37.186783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e7de7-3dd2-4ce5-b0b8-6f5bc4888260 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:37.186865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e7de7-3dd2-4ce5-b0b8-6f5bc4888260 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:37.188197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 29/58] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:37.193363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77927e55-27ba-472b-bbbf-0080b21bb10f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:37.194030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77927e55-27ba-472b-bbbf-0080b21bb10f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:37.194201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77927e55-27ba-472b-bbbf-0080b21bb10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:37.212597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-77927e55-27ba-472b-bbbf-0080b21bb10f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:37.213790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-77927e55-27ba-472b-bbbf-0080b21bb10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:37.283483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 30/59] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:37.555969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e7de7-3dd2-4ce5-b0b8-6f5bc4888260 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:37.654972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e7de7-3dd2-4ce5-b0b8-6f5bc4888260 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:37.677707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:37.689222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6ef03-d961-4a1e-a7ad-ab8d19d1888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:37.689860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ef03-d961-4a1e-a7ad-ab8d19d1888d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:37.690142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ef03-d961-4a1e-a7ad-ab8d19d1888d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:37.690417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ef03-d961-4a1e-a7ad-ab8d19d1888d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:37.691563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6ef03-d961-4a1e-a7ad-ab8d19d1888d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:37.692988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6ef03-d961-4a1e-a7ad-ab8d19d1888d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:37.694824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 30/60] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:37.700232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77927e55-27ba-472b-bbbf-0080b21bb10f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:37.795733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77927e55-27ba-472b-bbbf-0080b21bb10f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:37.824403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:37.830449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e28544-bb58-49fd-a3c3-d64eb0250b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:37.831099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e28544-bb58-49fd-a3c3-d64eb0250b61 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:37.831376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e28544-bb58-49fd-a3c3-d64eb0250b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:37.831644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e28544-bb58-49fd-a3c3-d64eb0250b61 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:37.832297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e28544-bb58-49fd-a3c3-d64eb0250b61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:37.833723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e28544-bb58-49fd-a3c3-d64eb0250b61 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:37.858936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 31/61] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:47:37 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:47:40.659417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e611ff1-a36d-47d3-b906-135b791e4ce2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:40.660386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e611ff1-a36d-47d3-b906-135b791e4ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:40.660879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e611ff1-a36d-47d3-b906-135b791e4ce2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:40.662686 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 31/62] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:40.667752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc62e-bda3-47bf-8577-fb3cf6755871 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:40.668541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc62e-bda3-47bf-8577-fb3cf6755871 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:40.669018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc62e-bda3-47bf-8577-fb3cf6755871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:40.704541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-536dc62e-bda3-47bf-8577-fb3cf6755871 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:40.705747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-536dc62e-bda3-47bf-8577-fb3cf6755871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:40.801293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 32/63] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:40.974450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bb0bc-d1ff-4d96-a878-5a50b2677190 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:40.974450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bb0bc-d1ff-4d96-a878-5a50b2677190 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:40.974450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bb0bc-d1ff-4d96-a878-5a50b2677190 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:40.975159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 32/64] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:40.979490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897942f4-d59b-4040-894d-645a8397a5c5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:40.979981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897942f4-d59b-4040-894d-645a8397a5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:40.980442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897942f4-d59b-4040-894d-645a8397a5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.008087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-897942f4-d59b-4040-894d-645a8397a5c5 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:41.009651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-897942f4-d59b-4040-894d-645a8397a5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:41.084137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3bb0bc-d1ff-4d96-a878-5a50b2677190 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:41.106474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 33/65] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:41.224318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3bb0bc-d1ff-4d96-a878-5a50b2677190 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:41.273593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c68954-2c2f-4fee-bcf9-6892b1ef264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:41.274420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c68954-2c2f-4fee-bcf9-6892b1ef264c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:41.274706 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c68954-2c2f-4fee-bcf9-6892b1ef264c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:41.274982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c68954-2c2f-4fee-bcf9-6892b1ef264c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.276603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f9468-06a0-4bf6-86ee-3c3ff991e728 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:41.277307 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f9468-06a0-4bf6-86ee-3c3ff991e728 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:41.277784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f9468-06a0-4bf6-86ee-3c3ff991e728 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.279926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 34/66] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:47:41.285214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472a710-dc8e-46fd-86c6-78a23c521706 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:41.285737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472a710-dc8e-46fd-86c6-78a23c521706 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:41.286346 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472a710-dc8e-46fd-86c6-78a23c521706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.291411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c68954-2c2f-4fee-bcf9-6892b1ef264c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-54c68954-2c2f-4fee-bcf9-6892b1ef264c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:41.298185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:41.299058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:41.299815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:41.312775 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 33/67] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:41.315777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c68954-2c2f-4fee-bcf9-6892b1ef264c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:41.330315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9472a710-dc8e-46fd-86c6-78a23c521706 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:41.332833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9472a710-dc8e-46fd-86c6-78a23c521706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:41.413285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:41.417178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:41.448762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 35/68] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:41.450465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472a710-dc8e-46fd-86c6-78a23c521706 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:41.480785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c68954-2c2f-4fee-bcf9-6892b1ef264c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:41.550216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f74977-104b-4845-a01d-cd4917e245f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:41.551213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f74977-104b-4845-a01d-cd4917e245f2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:41.551689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f74977-104b-4845-a01d-cd4917e245f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:41.552151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f74977-104b-4845-a01d-cd4917e245f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.553004 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f74977-104b-4845-a01d-cd4917e245f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:41.553497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f74977-104b-4845-a01d-cd4917e245f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:41.555107 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f74977-104b-4845-a01d-cd4917e245f2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:41.577293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 34/69] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:47:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:41.583489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f74977-104b-4845-a01d-cd4917e245f2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:41.636800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472a710-dc8e-46fd-86c6-78a23c521706 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:41.675273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206b425-32fc-48ad-8736-18ca667951a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:41.675922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206b425-32fc-48ad-8736-18ca667951a0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:41.676183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206b425-32fc-48ad-8736-18ca667951a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:41.676623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206b425-32fc-48ad-8736-18ca667951a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.687346 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206b425-32fc-48ad-8736-18ca667951a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c206b425-32fc-48ad-8736-18ca667951a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:41.692214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:41.693062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 36/70] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:47:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:41.699441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f2099-0340-40d6-a9bd-cb60aa733394 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:41.700108 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f2099-0340-40d6-a9bd-cb60aa733394 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:41.700108 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f2099-0340-40d6-a9bd-cb60aa733394 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.700108 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 37/71] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:41.702678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250993-1829-4f46-acd4-a1cd2752e6e8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:41.703506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250993-1829-4f46-acd4-a1cd2752e6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:41.703790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250993-1829-4f46-acd4-a1cd2752e6e8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.705024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 38/72] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:41.705791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7250993-1829-4f46-acd4-a1cd2752e6e8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:41.733490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f74977-104b-4845-a01d-cd4917e245f2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:41.745122 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:41.748692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:41.771597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4f5db-a7f6-478b-b79a-bb8becacd098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:41.772276 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4f5db-a7f6-478b-b79a-bb8becacd098 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:41.772552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4f5db-a7f6-478b-b79a-bb8becacd098 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:41.772848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4f5db-a7f6-478b-b79a-bb8becacd098 None admin] PATH_INFO: `/v3/users/216c0adc13b34a498d52b72c7b75c72b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.788052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4f5db-a7f6-478b-b79a-bb8becacd098 None admin] RBAC: Authorizing `identity:get_user(user_id=216c0adc13b34a498d52b72c7b75c72b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:41.789600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4f5db-a7f6-478b-b79a-bb8becacd098 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:41.791321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 35/73] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:41 2022] GET /identity/v3/users/216c0adc13b34a498d52b72c7b75c72b => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:41.792615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aac8a-244e-4280-9cc7-40dfd457f480 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:41.792903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aac8a-244e-4280-9cc7-40dfd457f480 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:41.793198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aac8a-244e-4280-9cc7-40dfd457f480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.812879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-043aac8a-244e-4280-9cc7-40dfd457f480 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:41.814060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-043aac8a-244e-4280-9cc7-40dfd457f480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:41.815984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7250993-1829-4f46-acd4-a1cd2752e6e8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:41.849808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3827c32-3131-4d16-b9a4-752ecfc6dec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:41.850411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3827c32-3131-4d16-b9a4-752ecfc6dec1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:41.850691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3827c32-3131-4d16-b9a4-752ecfc6dec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:41.850958 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3827c32-3131-4d16-b9a4-752ecfc6dec1 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.851642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3827c32-3131-4d16-b9a4-752ecfc6dec1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:41.851943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3827c32-3131-4d16-b9a4-752ecfc6dec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:41.853326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3827c32-3131-4d16-b9a4-752ecfc6dec1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:41.861274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 39/74] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:41.862423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905ae5d-48a4-4a0d-bbd4-2459eebfd4fc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:41.862691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905ae5d-48a4-4a0d-bbd4-2459eebfd4fc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:41.862966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905ae5d-48a4-4a0d-bbd4-2459eebfd4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:41.880625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9905ae5d-48a4-4a0d-bbd4-2459eebfd4fc None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:41.881495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 36/75] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:41.881833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9905ae5d-48a4-4a0d-bbd4-2459eebfd4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:41.882330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043aac8a-244e-4280-9cc7-40dfd457f480 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:41.965056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 40/76] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:41.966348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905ae5d-48a4-4a0d-bbd4-2459eebfd4fc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:41.992638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043aac8a-244e-4280-9cc7-40dfd457f480 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:41.994498 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:42.005930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:42.040855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04f430-45bd-46b9-8f7c-b73fcea09fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:42.041539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f430-45bd-46b9-8f7c-b73fcea09fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:42.041862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f430-45bd-46b9-8f7c-b73fcea09fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:42.042139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f430-45bd-46b9-8f7c-b73fcea09fd8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:42.057438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04f430-45bd-46b9-8f7c-b73fcea09fd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0f04f430-45bd-46b9-8f7c-b73fcea09fd8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:42.064130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:42.064952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:42.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 37/77] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:47:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:42.066820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04f430-45bd-46b9-8f7c-b73fcea09fd8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:42.101021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905ae5d-48a4-4a0d-bbd4-2459eebfd4fc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:42.139023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debbc12-809d-4373-b890-1a4d83bf7260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:42.139949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debbc12-809d-4373-b890-1a4d83bf7260 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:42.140496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debbc12-809d-4373-b890-1a4d83bf7260 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:42.140985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debbc12-809d-4373-b890-1a4d83bf7260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:42.196323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04f430-45bd-46b9-8f7c-b73fcea09fd8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:42.211107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debbc12-809d-4373-b890-1a4d83bf7260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:42.212648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debbc12-809d-4373-b890-1a4d83bf7260 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:42.231947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7795c-65ac-4b49-9f23-766655490a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:42.232850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7795c-65ac-4b49-9f23-766655490a3f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:42.233350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7795c-65ac-4b49-9f23-766655490a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:42.233815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7795c-65ac-4b49-9f23-766655490a3f None admin] PATH_INFO: `/v3/users/9c8f433f2f1f4923859810dc0019845c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:42.235659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7795c-65ac-4b49-9f23-766655490a3f None admin] RBAC: Authorizing `identity:get_user(user_id=9c8f433f2f1f4923859810dc0019845c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:42.237710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7795c-65ac-4b49-9f23-766655490a3f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:42.239700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:42.240409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 38/78] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:42 2022] GET /identity/v3/users/9c8f433f2f1f4923859810dc0019845c => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:42.241445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7795c-65ac-4b49-9f23-766655490a3f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:42.275542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:42.320268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 41/79] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:47:42.321146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debbc12-809d-4373-b890-1a4d83bf7260 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:42.346533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7795c-65ac-4b49-9f23-766655490a3f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:42.381526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ed6d6-a42f-4f90-b15c-904e32144490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:42.382130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ed6d6-a42f-4f90-b15c-904e32144490 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:42.382409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ed6d6-a42f-4f90-b15c-904e32144490 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:42.382683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ed6d6-a42f-4f90-b15c-904e32144490 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:42.383371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ed6d6-a42f-4f90-b15c-904e32144490 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:42.383672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ed6d6-a42f-4f90-b15c-904e32144490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:42.385049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ed6d6-a42f-4f90-b15c-904e32144490 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:42.393219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 39/80] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:47:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:42.394002 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ed6d6-a42f-4f90-b15c-904e32144490 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:42.424038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debbc12-809d-4373-b890-1a4d83bf7260 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:42.450252 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:42.460462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77052a4-0018-4c80-8e93-ce9ee2b78960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:42.461220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77052a4-0018-4c80-8e93-ce9ee2b78960 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:42.461529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77052a4-0018-4c80-8e93-ce9ee2b78960 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:42.461813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77052a4-0018-4c80-8e93-ce9ee2b78960 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:42.474214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77052a4-0018-4c80-8e93-ce9ee2b78960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c77052a4-0018-4c80-8e93-ce9ee2b78960 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:42.479049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:42.479974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 42/81] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:42.481730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77052a4-0018-4c80-8e93-ce9ee2b78960 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:42.487898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:42.497637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ed6d6-a42f-4f90-b15c-904e32144490 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:42.532580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68e2a5-5017-45ef-b102-224b2fbf48b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:42.533192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68e2a5-5017-45ef-b102-224b2fbf48b9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:42.533470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68e2a5-5017-45ef-b102-224b2fbf48b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:42.533880 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68e2a5-5017-45ef-b102-224b2fbf48b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:42.584443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77052a4-0018-4c80-8e93-ce9ee2b78960 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:42.599886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68e2a5-5017-45ef-b102-224b2fbf48b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:42.601375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68e2a5-5017-45ef-b102-224b2fbf48b9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:42.617763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6361df8-8196-4f99-9517-263a9a74fe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:42.618466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6361df8-8196-4f99-9517-263a9a74fe70 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:42.618754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6361df8-8196-4f99-9517-263a9a74fe70 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:42.619088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6361df8-8196-4f99-9517-263a9a74fe70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:42.620204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6361df8-8196-4f99-9517-263a9a74fe70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:42.621648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6361df8-8196-4f99-9517-263a9a74fe70 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:42.623437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 43/82] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:42.624000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6361df8-8196-4f99-9517-263a9a74fe70 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:42.680217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:42.695564 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:42.706243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 40/83] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:47:42.707115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68e2a5-5017-45ef-b102-224b2fbf48b9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:42.730932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6361df8-8196-4f99-9517-263a9a74fe70 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:42.767547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4041c1-ff74-491b-ac88-d69acfde35aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:42.768261 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4041c1-ff74-491b-ac88-d69acfde35aa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:42.768545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4041c1-ff74-491b-ac88-d69acfde35aa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:42.768825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4041c1-ff74-491b-ac88-d69acfde35aa None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:42.769953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4041c1-ff74-491b-ac88-d69acfde35aa None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:42.771435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4041c1-ff74-491b-ac88-d69acfde35aa None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:42.773087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 44/84] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:42 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:42.773854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4041c1-ff74-491b-ac88-d69acfde35aa None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:42.805876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68e2a5-5017-45ef-b102-224b2fbf48b9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:42.841667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e5517-57ea-4ac3-b006-55f96f24f461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:42.842222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e5517-57ea-4ac3-b006-55f96f24f461 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:42.842549 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e5517-57ea-4ac3-b006-55f96f24f461 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:42.842827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e5517-57ea-4ac3-b006-55f96f24f461 None admin] PATH_INFO: `/v3/users/216c0adc13b34a498d52b72c7b75c72b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:42.844025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e5517-57ea-4ac3-b006-55f96f24f461 None admin] RBAC: Authorizing `identity:get_user(user_id=216c0adc13b34a498d52b72c7b75c72b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:42.845473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e5517-57ea-4ac3-b006-55f96f24f461 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:42.847432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 41/85] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:42 2022] GET /identity/v3/users/216c0adc13b34a498d52b72c7b75c72b => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:42.848071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e5517-57ea-4ac3-b006-55f96f24f461 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:42.877565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4041c1-ff74-491b-ac88-d69acfde35aa None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:42.900601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:42.908119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:42.913409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c233f0-456c-4aab-a603-38e035227339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:42.914020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c233f0-456c-4aab-a603-38e035227339 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:42.914302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c233f0-456c-4aab-a603-38e035227339 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:42.914584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c233f0-456c-4aab-a603-38e035227339 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:42.915278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c233f0-456c-4aab-a603-38e035227339 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:42.915630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c233f0-456c-4aab-a603-38e035227339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:42.917010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c233f0-456c-4aab-a603-38e035227339 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:42.925591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 45/86] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:47:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:42.926386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c233f0-456c-4aab-a603-38e035227339 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:42.953635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e5517-57ea-4ac3-b006-55f96f24f461 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:42.987683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee4a5a-0f8a-412a-8c06-9e4e696d24ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:42.988298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4a5a-0f8a-412a-8c06-9e4e696d24ae None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:42.988584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4a5a-0f8a-412a-8c06-9e4e696d24ae None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:42.988862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4a5a-0f8a-412a-8c06-9e4e696d24ae None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:42.989525 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee4a5a-0f8a-412a-8c06-9e4e696d24ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:42.990958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee4a5a-0f8a-412a-8c06-9e4e696d24ae None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:43.019126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 42/87] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:47:42 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:47:43.019960 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee4a5a-0f8a-412a-8c06-9e4e696d24ae None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:43.029350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c233f0-456c-4aab-a603-38e035227339 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:43.069763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69829c-535c-41ff-affd-e4ca5fd23e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:43.070440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69829c-535c-41ff-affd-e4ca5fd23e42 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:43.070714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69829c-535c-41ff-affd-e4ca5fd23e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:43.071033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69829c-535c-41ff-affd-e4ca5fd23e42 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:43.072267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69829c-535c-41ff-affd-e4ca5fd23e42 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:43.073785 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69829c-535c-41ff-affd-e4ca5fd23e42 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:43.075768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 46/88] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:42 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:43.076513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69829c-535c-41ff-affd-e4ca5fd23e42 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:43.138397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee4a5a-0f8a-412a-8c06-9e4e696d24ae None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:43.143807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:43.159815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:43.181541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2a66d-861b-4e51-85c9-483687a9395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:43.182166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2a66d-861b-4e51-85c9-483687a9395f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:43.182450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2a66d-861b-4e51-85c9-483687a9395f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:43.182725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2a66d-861b-4e51-85c9-483687a9395f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:43.183507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2a66d-861b-4e51-85c9-483687a9395f None admin] RBAC: Adding query filter params (role.id=a0db5bd603fc4096917bbcd8c6932bc0, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086, user.id=216c0adc13b34a498d52b72c7b75c72b) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:43.183825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2a66d-861b-4e51-85c9-483687a9395f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:43.185299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2a66d-861b-4e51-85c9-483687a9395f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:43.194177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 43/89] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:43 2022] GET /identity/v3/role_assignments?user.id=216c0adc13b34a498d52b72c7b75c72b&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:43.195645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2a66d-861b-4e51-85c9-483687a9395f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:43.202664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69829c-535c-41ff-affd-e4ca5fd23e42 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:43.261432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f1b3e-e2cb-4a52-b5d9-b712303683c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:43.262097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f1b3e-e2cb-4a52-b5d9-b712303683c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:43.262819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f1b3e-e2cb-4a52-b5d9-b712303683c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:43.262819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f1b3e-e2cb-4a52-b5d9-b712303683c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:43.345873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2a66d-861b-4e51-85c9-483687a9395f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:43.359055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f1b3e-e2cb-4a52-b5d9-b712303683c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:43.360563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f1b3e-e2cb-4a52-b5d9-b712303683c4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:43.387892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b12c8-3d06-44b1-9320-60cc17efe32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:43.388598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b12c8-3d06-44b1-9320-60cc17efe32c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:43.388827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b12c8-3d06-44b1-9320-60cc17efe32c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:43.389105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b12c8-3d06-44b1-9320-60cc17efe32c None admin] PATH_INFO: `/v3/users/216c0adc13b34a498d52b72c7b75c72b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:43.390255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b12c8-3d06-44b1-9320-60cc17efe32c None admin] RBAC: Authorizing `identity:get_user(user_id=216c0adc13b34a498d52b72c7b75c72b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:43.392404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b12c8-3d06-44b1-9320-60cc17efe32c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:43.394180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 44/90] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:43 2022] GET /identity/v3/users/216c0adc13b34a498d52b72c7b75c72b => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:43.398905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b12c8-3d06-44b1-9320-60cc17efe32c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:43.447811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:43.457334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:43.511671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 47/91] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:47:43.512944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f1b3e-e2cb-4a52-b5d9-b712303683c4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:43.523605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b12c8-3d06-44b1-9320-60cc17efe32c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:43.571779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662ad4e-5e73-49fd-93b3-c89b652666e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:43.573589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662ad4e-5e73-49fd-93b3-c89b652666e1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:43.574255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662ad4e-5e73-49fd-93b3-c89b652666e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:43.576002 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662ad4e-5e73-49fd-93b3-c89b652666e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:43.577693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662ad4e-5e73-49fd-93b3-c89b652666e1 None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086, user.id=9c8f433f2f1f4923859810dc0019845c) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:43.581605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662ad4e-5e73-49fd-93b3-c89b652666e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:43.583422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662ad4e-5e73-49fd-93b3-c89b652666e1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:43.592452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 45/92] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:43 2022] GET /identity/v3/role_assignments?user.id=9c8f433f2f1f4923859810dc0019845c&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:43.594209 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662ad4e-5e73-49fd-93b3-c89b652666e1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:43.661360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f1b3e-e2cb-4a52-b5d9-b712303683c4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:43.737370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d21ba-21f9-4a67-bdbc-b3eef73f6837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:43.738827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d21ba-21f9-4a67-bdbc-b3eef73f6837 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:43.739289 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d21ba-21f9-4a67-bdbc-b3eef73f6837 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:43.739711 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d21ba-21f9-4a67-bdbc-b3eef73f6837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:43.779148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662ad4e-5e73-49fd-93b3-c89b652666e1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:43.789783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:43.793678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:43.823763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a35a3-1aac-472d-a1df-a71421527c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:43.824425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a35a3-1aac-472d-a1df-a71421527c09 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:43.824714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a35a3-1aac-472d-a1df-a71421527c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:43.824993 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a35a3-1aac-472d-a1df-a71421527c09 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:43.831817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a35a3-1aac-472d-a1df-a71421527c09 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:43.833315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a35a3-1aac-472d-a1df-a71421527c09 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:43.835044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 46/93] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:43 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:43.840921 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a35a3-1aac-472d-a1df-a71421527c09 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:43.886950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d21ba-21f9-4a67-bdbc-b3eef73f6837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:43.888816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d21ba-21f9-4a67-bdbc-b3eef73f6837 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:43.943894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a35a3-1aac-472d-a1df-a71421527c09 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:44.003745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23f499-ad98-4300-b76e-214cf923ca2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:44.004379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23f499-ad98-4300-b76e-214cf923ca2c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:44.004665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23f499-ad98-4300-b76e-214cf923ca2c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:44.004936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23f499-ad98-4300-b76e-214cf923ca2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:44.005667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23f499-ad98-4300-b76e-214cf923ca2c None admin] RBAC: Adding query filter params (role.id=31cc9f3a35594d40924277f4778c9f03, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086, user.id=216c0adc13b34a498d52b72c7b75c72b) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:44.005977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23f499-ad98-4300-b76e-214cf923ca2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:44.007459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23f499-ad98-4300-b76e-214cf923ca2c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:44.013422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 47/94] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:43 2022] GET /identity/v3/role_assignments?user.id=216c0adc13b34a498d52b72c7b75c72b&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=31cc9f3a35594d40924277f4778c9f03 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:44.024020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 48/95] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:47:44.032962 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23f499-ad98-4300-b76e-214cf923ca2c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:44.118907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:44.188625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23f499-ad98-4300-b76e-214cf923ca2c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:44.224501 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0588c2-12b1-4d3b-b520-6d1a6f5cebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:44.225064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0588c2-12b1-4d3b-b520-6d1a6f5cebf9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:44.225342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0588c2-12b1-4d3b-b520-6d1a6f5cebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:44.225616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0588c2-12b1-4d3b-b520-6d1a6f5cebf9 None admin] PATH_INFO: `/v3/projects/e8eb80afcc8c40d6bfae07f43cecacc8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:44.226752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0588c2-12b1-4d3b-b520-6d1a6f5cebf9 None admin] RBAC: Authorizing `identity:get_project(project_id=e8eb80afcc8c40d6bfae07f43cecacc8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:44.228245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0588c2-12b1-4d3b-b520-6d1a6f5cebf9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:44.230082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 48/96] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:44 2022] GET /identity/v3/projects/e8eb80afcc8c40d6bfae07f43cecacc8 => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:44.235982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d21ba-21f9-4a67-bdbc-b3eef73f6837 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:44.342600 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:44.405571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d21ba-21f9-4a67-bdbc-b3eef73f6837 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:44.459695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4efe7-a3cd-495e-9262-59e6e932cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:44.460314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4efe7-a3cd-495e-9262-59e6e932cc56 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:44.460597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4efe7-a3cd-495e-9262-59e6e932cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:44.460875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4efe7-a3cd-495e-9262-59e6e932cc56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:44.461603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4efe7-a3cd-495e-9262-59e6e932cc56 None admin] RBAC: Adding query filter params (role.id=a0db5bd603fc4096917bbcd8c6932bc0, scope.project.id=e8eb80afcc8c40d6bfae07f43cecacc8, user.id=216c0adc13b34a498d52b72c7b75c72b) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:44.461920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4efe7-a3cd-495e-9262-59e6e932cc56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:44.463395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4efe7-a3cd-495e-9262-59e6e932cc56 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:44.475114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 49/97] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:44 2022] GET /identity/v3/role_assignments?user.id=216c0adc13b34a498d52b72c7b75c72b&scope.project.id=e8eb80afcc8c40d6bfae07f43cecacc8&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:45.851086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1b99e-f396-4a42-8413-f6ed396365bf None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:45.851616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1b99e-f396-4a42-8413-f6ed396365bf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:45.851616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1b99e-f396-4a42-8413-f6ed396365bf None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:45.852976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 49/98] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:45.870591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d597209-58bb-4022-8cae-71d6112853d7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:45.871069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d597209-58bb-4022-8cae-71d6112853d7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:45.871173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d597209-58bb-4022-8cae-71d6112853d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:45.886925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0f75a-d9c8-4fec-992e-b41cb9deb0c5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:45.887239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0f75a-d9c8-4fec-992e-b41cb9deb0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:45.887544 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0f75a-d9c8-4fec-992e-b41cb9deb0c5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:45.888830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 50/99] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:47:45.894692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba823-54a6-4666-b319-c491fa11d433 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:45.894988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba823-54a6-4666-b319-c491fa11d433 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:45.895307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba823-54a6-4666-b319-c491fa11d433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:45.903574 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4d597209-58bb-4022-8cae-71d6112853d7 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:45.904795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4d597209-58bb-4022-8cae-71d6112853d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:45.925660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-86fba823-54a6-4666-b319-c491fa11d433 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:45.926869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-86fba823-54a6-4666-b319-c491fa11d433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:46.038708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 50/100] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:46.040532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ce588-ee7e-4254-aa13-4065b4e54dde None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:46.040826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ce588-ee7e-4254-aa13-4065b4e54dde None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:46.041103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ce588-ee7e-4254-aa13-4065b4e54dde None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:46.042567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 51/101] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:46.062920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04d91a-f0e2-4b6b-bce0-a2b7189adb69 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:46.063246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04d91a-f0e2-4b6b-bce0-a2b7189adb69 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:46.063523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04d91a-f0e2-4b6b-bce0-a2b7189adb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:46.076345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 51/102] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:46.099506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2e04d91a-f0e2-4b6b-bce0-a2b7189adb69 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:46.100796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2e04d91a-f0e2-4b6b-bce0-a2b7189adb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:46.162573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:46.195345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 52/103] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:46.416569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fba823-54a6-4666-b319-c491fa11d433 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:46.473547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:46.536172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04d91a-f0e2-4b6b-bce0-a2b7189adb69 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:46.566861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fba823-54a6-4666-b319-c491fa11d433 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:46.642845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04d91a-f0e2-4b6b-bce0-a2b7189adb69 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:46.652466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbd896-854b-4642-8e06-bc957de6aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:46.653497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbd896-854b-4642-8e06-bc957de6aed0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:46.654006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbd896-854b-4642-8e06-bc957de6aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:46.654462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbd896-854b-4642-8e06-bc957de6aed0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:46.690115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e462878-7917-4c0b-a65f-434029a050ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:46.690978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e462878-7917-4c0b-a65f-434029a050ae None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:46.690978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e462878-7917-4c0b-a65f-434029a050ae None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:46.695449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e462878-7917-4c0b-a65f-434029a050ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:46.698186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbd896-854b-4642-8e06-bc957de6aed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-bccbd896-854b-4642-8e06-bc957de6aed0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:46.705111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:46.706110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:47:46.706872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:46.709004 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e462878-7917-4c0b-a65f-434029a050ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:46.715086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 52/104] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:46.720044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbd896-854b-4642-8e06-bc957de6aed0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5e462878-7917-4c0b-a65f-434029a050ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:46.722822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:46.724550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:47:46.726468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 53/105] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:47:46.730087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e462878-7917-4c0b-a65f-434029a050ae None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:46.809693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:46.858537 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbd896-854b-4642-8e06-bc957de6aed0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:46.861122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:46.892250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e462878-7917-4c0b-a65f-434029a050ae None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:46.899133 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265003b-c57d-40f4-9225-e510f67b5454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:46.900049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265003b-c57d-40f4-9225-e510f67b5454 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:46.900532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265003b-c57d-40f4-9225-e510f67b5454 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:46.901057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265003b-c57d-40f4-9225-e510f67b5454 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:46.901939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265003b-c57d-40f4-9225-e510f67b5454 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:46.902320 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265003b-c57d-40f4-9225-e510f67b5454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:46.904126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265003b-c57d-40f4-9225-e510f67b5454 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:46.913406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 53/106] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:47:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:46.914124 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265003b-c57d-40f4-9225-e510f67b5454 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:46.938454 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8485547-ffd7-4f1d-8be4-fc69ab3528e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:46.939707 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8485547-ffd7-4f1d-8be4-fc69ab3528e2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:46.940251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8485547-ffd7-4f1d-8be4-fc69ab3528e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:46.940941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8485547-ffd7-4f1d-8be4-fc69ab3528e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:46.942275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8485547-ffd7-4f1d-8be4-fc69ab3528e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:46.943013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8485547-ffd7-4f1d-8be4-fc69ab3528e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:46.945558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8485547-ffd7-4f1d-8be4-fc69ab3528e2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:46.958960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 54/107] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:47:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:46.961476 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15310f8b-1f7f-4b99-b439-2df32ce522d2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:46.962196 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15310f8b-1f7f-4b99-b439-2df32ce522d2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:46.962643 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15310f8b-1f7f-4b99-b439-2df32ce522d2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:46.969209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 55/108] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:46.969209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15310f8b-1f7f-4b99-b439-2df32ce522d2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:47.028938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265003b-c57d-40f4-9225-e510f67b5454 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:47.075650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3392f13-2707-4a95-94e5-42207d3237da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:47.076616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3392f13-2707-4a95-94e5-42207d3237da None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.077144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3392f13-2707-4a95-94e5-42207d3237da None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.077747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3392f13-2707-4a95-94e5-42207d3237da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.079169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3392f13-2707-4a95-94e5-42207d3237da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:47.080927 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3392f13-2707-4a95-94e5-42207d3237da None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:47.082874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 54/109] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:47.083950 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3392f13-2707-4a95-94e5-42207d3237da None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:47.103401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:47.110080 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15310f8b-1f7f-4b99-b439-2df32ce522d2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:47.126155 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:47.160727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa53754-99af-4d32-b377-872d285c3ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:47.162473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa53754-99af-4d32-b377-872d285c3ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.163157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa53754-99af-4d32-b377-872d285c3ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.165200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa53754-99af-4d32-b377-872d285c3ab6 None admin] PATH_INFO: `/v3/users/fbfa2ddf17434fa09745d5a1fa32cb4f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.186908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa53754-99af-4d32-b377-872d285c3ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=fbfa2ddf17434fa09745d5a1fa32cb4f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:47.188928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa53754-99af-4d32-b377-872d285c3ab6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:47.192283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 56/110] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:46 2022] GET /identity/v3/users/fbfa2ddf17434fa09745d5a1fa32cb4f => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:47.192852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f554f97-904d-468b-9463-df73de85d104 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.193144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f554f97-904d-468b-9463-df73de85d104 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.193422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f554f97-904d-468b-9463-df73de85d104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.214880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2f554f97-904d-468b-9463-df73de85d104 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:47.216264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2f554f97-904d-468b-9463-df73de85d104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:47.216659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3392f13-2707-4a95-94e5-42207d3237da None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:47.254459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c3331-c6ee-43d7-81e3-72cf68a252ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:47.255553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c3331-c6ee-43d7-81e3-72cf68a252ff None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.256029 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c3331-c6ee-43d7-81e3-72cf68a252ff None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.256562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c3331-c6ee-43d7-81e3-72cf68a252ff None admin] PATH_INFO: `/v3/users/216c0adc13b34a498d52b72c7b75c72b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.257891 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c3331-c6ee-43d7-81e3-72cf68a252ff None admin] RBAC: Authorizing `identity:get_user(user_id=216c0adc13b34a498d52b72c7b75c72b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:47.259713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c3331-c6ee-43d7-81e3-72cf68a252ff None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:47.261562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 55/111] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:47 2022] GET /identity/v3/users/216c0adc13b34a498d52b72c7b75c72b => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:47.262692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c3331-c6ee-43d7-81e3-72cf68a252ff None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:47.295022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 57/112] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:47.295022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f554f97-904d-468b-9463-df73de85d104 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:47.383105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:47.386214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c3331-c6ee-43d7-81e3-72cf68a252ff None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:47.388169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:47.403968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f554f97-904d-468b-9463-df73de85d104 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:47.423581 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b9244-7a4d-45d2-b55e-05a3c1abd226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:47.424200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b9244-7a4d-45d2-b55e-05a3c1abd226 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.424482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b9244-7a4d-45d2-b55e-05a3c1abd226 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.424757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b9244-7a4d-45d2-b55e-05a3c1abd226 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.425402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b9244-7a4d-45d2-b55e-05a3c1abd226 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:47.426864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b9244-7a4d-45d2-b55e-05a3c1abd226 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:47.442603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffe864-6f25-411c-8088-71cdf34ba84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:47.442603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffe864-6f25-411c-8088-71cdf34ba84f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.442603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffe864-6f25-411c-8088-71cdf34ba84f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.442603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffe864-6f25-411c-8088-71cdf34ba84f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.462518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 56/113] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:47:47 2022] POST /identity/v3/projects => generated 302 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:47:47.463468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b9244-7a4d-45d2-b55e-05a3c1abd226 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:47.541210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffe864-6f25-411c-8088-71cdf34ba84f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:47.542875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffe864-6f25-411c-8088-71cdf34ba84f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:47.606445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b9244-7a4d-45d2-b55e-05a3c1abd226 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:47.648920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:47.650125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa1475-056a-4161-bb31-523e61dda4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:47.650954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1475-056a-4161-bb31-523e61dda4ae None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.651273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1475-056a-4161-bb31-523e61dda4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.651552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1475-056a-4161-bb31-523e61dda4ae None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.652686 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa1475-056a-4161-bb31-523e61dda4ae None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:47.654605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa1475-056a-4161-bb31-523e61dda4ae None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:47.656858 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 57/114] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:47 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:47.658187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f1f29-8ea5-430b-af12-529ece3add6f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.658573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f1f29-8ea5-430b-af12-529ece3add6f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.659039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f1f29-8ea5-430b-af12-529ece3add6f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.660282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 58/115] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:47.661251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c6afe-d062-4029-96be-2ed381fbfc5b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.661539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c6afe-d062-4029-96be-2ed381fbfc5b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.661816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c6afe-d062-4029-96be-2ed381fbfc5b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.663174 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 59/116] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:47.664034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c6afe-d062-4029-96be-2ed381fbfc5b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:47.669240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 58/117] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:47:47.672000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58435e5a-82f7-4898-b684-19d18cc786a7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.672750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58435e5a-82f7-4898-b684-19d18cc786a7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.673530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58435e5a-82f7-4898-b684-19d18cc786a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.688046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:47.699822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-58435e5a-82f7-4898-b684-19d18cc786a7 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:47.700945 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-58435e5a-82f7-4898-b684-19d18cc786a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:47.773893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c6afe-d062-4029-96be-2ed381fbfc5b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:47.783032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 59/118] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:47.783032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae404-75e0-409a-bb65-2604c6b27eb0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.783032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae404-75e0-409a-bb65-2604c6b27eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.783032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae404-75e0-409a-bb65-2604c6b27eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.802017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a86ae404-75e0-409a-bb65-2604c6b27eb0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:47.803348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a86ae404-75e0-409a-bb65-2604c6b27eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:47.821950 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bd26f-a497-4488-b84c-c877e4739d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:47.825380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd26f-a497-4488-b84c-c877e4739d82 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:47.825380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd26f-a497-4488-b84c-c877e4739d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:47.825550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd26f-a497-4488-b84c-c877e4739d82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:47.837317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bd26f-a497-4488-b84c-c877e4739d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-722bd26f-a497-4488-b84c-c877e4739d82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:47.842216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:47.843087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:47.844399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 60/119] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:47:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:47.845049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bd26f-a497-4488-b84c-c877e4739d82 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:47.910896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 60/120] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:47.912379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ae404-75e0-409a-bb65-2604c6b27eb0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:47.938660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:47.964392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bd26f-a497-4488-b84c-c877e4739d82 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:47.970586 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:48.001419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6dfac-b2fe-4069-a951-175314316e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:48.002144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6dfac-b2fe-4069-a951-175314316e11 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:48.002411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6dfac-b2fe-4069-a951-175314316e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:48.002714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6dfac-b2fe-4069-a951-175314316e11 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/users/216c0adc13b34a498d52b72c7b75c72b/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:48.004722 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6dfac-b2fe-4069-a951-175314316e11 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdbff3dc80ab4433ba8d78b52eb66086, user_id=216c0adc13b34a498d52b72c7b75c72b, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:48.006174 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6dfac-b2fe-4069-a951-175314316e11 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:48.025614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 61/121] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:47:47 2022] PUT /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/users/216c0adc13b34a498d52b72c7b75c72b/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:47:48.026452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6dfac-b2fe-4069-a951-175314316e11 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:48.028001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ae404-75e0-409a-bb65-2604c6b27eb0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:48.120222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5de591-15a3-4b2b-bf33-770c27612923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:48.120970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de591-15a3-4b2b-bf33-770c27612923 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:48.121329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de591-15a3-4b2b-bf33-770c27612923 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:48.121609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de591-15a3-4b2b-bf33-770c27612923 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:48.122743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5de591-15a3-4b2b-bf33-770c27612923 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:48.124243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5de591-15a3-4b2b-bf33-770c27612923 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:48.125996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 61/122] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:47 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:48.132323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5de591-15a3-4b2b-bf33-770c27612923 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:48.168726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6dfac-b2fe-4069-a951-175314316e11 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:48.212979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcec208-919e-48e6-8997-4de7a8844451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:48.213649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcec208-919e-48e6-8997-4de7a8844451 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:48.213937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcec208-919e-48e6-8997-4de7a8844451 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:48.214217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcec208-919e-48e6-8997-4de7a8844451 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:48.214948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcec208-919e-48e6-8997-4de7a8844451 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:48.215377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcec208-919e-48e6-8997-4de7a8844451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:48.217035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcec208-919e-48e6-8997-4de7a8844451 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:48.228911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 62/123] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:48.236336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcec208-919e-48e6-8997-4de7a8844451 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:48.239627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:48.303964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5de591-15a3-4b2b-bf33-770c27612923 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:48.322716 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:48.351982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3c074-a8f3-467a-9e20-951d43de711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:48.352597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c074-a8f3-467a-9e20-951d43de711d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:48.352877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c074-a8f3-467a-9e20-951d43de711d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:48.353151 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c074-a8f3-467a-9e20-951d43de711d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:48.353875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3c074-a8f3-467a-9e20-951d43de711d None admin] RBAC: Adding query filter params (role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086, user.id=fbfa2ddf17434fa09745d5a1fa32cb4f) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:48.354182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3c074-a8f3-467a-9e20-951d43de711d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:48.355807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3c074-a8f3-467a-9e20-951d43de711d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:48.363444 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 62/124] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:48 2022] GET /identity/v3/role_assignments?user.id=fbfa2ddf17434fa09745d5a1fa32cb4f&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:48.365906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3c074-a8f3-467a-9e20-951d43de711d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:48.379237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcec208-919e-48e6-8997-4de7a8844451 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:48.417122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159ae44-f5bd-4dc3-b4fa-0b8185e3efb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:48.417774 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159ae44-f5bd-4dc3-b4fa-0b8185e3efb7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:48.418057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159ae44-f5bd-4dc3-b4fa-0b8185e3efb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:48.418360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159ae44-f5bd-4dc3-b4fa-0b8185e3efb7 None admin] PATH_INFO: `/v3/users/9c8f433f2f1f4923859810dc0019845c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:48.419592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159ae44-f5bd-4dc3-b4fa-0b8185e3efb7 None admin] RBAC: Authorizing `identity:get_user(user_id=9c8f433f2f1f4923859810dc0019845c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:48.421187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159ae44-f5bd-4dc3-b4fa-0b8185e3efb7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:48.422959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 63/125] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:48 2022] GET /identity/v3/users/9c8f433f2f1f4923859810dc0019845c => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:48.423872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159ae44-f5bd-4dc3-b4fa-0b8185e3efb7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:48.505672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:48.541789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159ae44-f5bd-4dc3-b4fa-0b8185e3efb7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:48.582199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43277fee-79fc-4083-93ad-2fb73c10ddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:48.582841 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43277fee-79fc-4083-93ad-2fb73c10ddb0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:48.583167 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43277fee-79fc-4083-93ad-2fb73c10ddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:48.583445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43277fee-79fc-4083-93ad-2fb73c10ddb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:48.595574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43277fee-79fc-4083-93ad-2fb73c10ddb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:48.597431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3c074-a8f3-467a-9e20-951d43de711d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-43277fee-79fc-4083-93ad-2fb73c10ddb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:48.601186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:48.602039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:48.602805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:48.608013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 64/126] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:48.608961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43277fee-79fc-4083-93ad-2fb73c10ddb0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:48.675186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac89c5-6263-48bc-8826-c41de9667b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:48.675861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac89c5-6263-48bc-8826-c41de9667b19 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:48.676106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac89c5-6263-48bc-8826-c41de9667b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:48.676378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac89c5-6263-48bc-8826-c41de9667b19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:48.706336 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac89c5-6263-48bc-8826-c41de9667b19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-32ac89c5-6263-48bc-8826-c41de9667b19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:48.712202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:48.713072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:47:48.713857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:47:48.721191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 63/127] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:47:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:48.724350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac89c5-6263-48bc-8826-c41de9667b19 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:48.769011 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43277fee-79fc-4083-93ad-2fb73c10ddb0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:48.803151 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:48.821558 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:48.834404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b3c56-8cb5-43cd-9daa-7683f3692c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:48.835410 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b3c56-8cb5-43cd-9daa-7683f3692c19 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:48.836941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b3c56-8cb5-43cd-9daa-7683f3692c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:48.837424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b3c56-8cb5-43cd-9daa-7683f3692c19 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:48.839470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b3c56-8cb5-43cd-9daa-7683f3692c19 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:48.841257 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b3c56-8cb5-43cd-9daa-7683f3692c19 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:48.843951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 65/128] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:48 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:48.848031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b3c56-8cb5-43cd-9daa-7683f3692c19 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:48.912760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac89c5-6263-48bc-8826-c41de9667b19 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:48.977163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cd635-c1d2-403a-8bac-745e6bb33197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:48.977782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cd635-c1d2-403a-8bac-745e6bb33197 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:48.978117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cd635-c1d2-403a-8bac-745e6bb33197 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:48.978399 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cd635-c1d2-403a-8bac-745e6bb33197 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:48.979092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002cd635-c1d2-403a-8bac-745e6bb33197 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:48.979403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002cd635-c1d2-403a-8bac-745e6bb33197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:48.980797 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002cd635-c1d2-403a-8bac-745e6bb33197 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:49.001385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 64/129] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:47:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:49.002954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cd635-c1d2-403a-8bac-745e6bb33197 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:49.046956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b3c56-8cb5-43cd-9daa-7683f3692c19 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:49.133484 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cd635-c1d2-403a-8bac-745e6bb33197 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:49.135432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66c785-e423-485c-988b-487fa66bef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:49.136327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66c785-e423-485c-988b-487fa66bef0a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:49.136453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:49.136924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66c785-e423-485c-988b-487fa66bef0a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:49.137397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66c785-e423-485c-988b-487fa66bef0a None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:49.138269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66c785-e423-485c-988b-487fa66bef0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:49.138770 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66c785-e423-485c-988b-487fa66bef0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:49.140892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66c785-e423-485c-988b-487fa66bef0a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:49.166136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 66/130] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:47:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:49.167325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66c785-e423-485c-988b-487fa66bef0a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:49.170066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b40f15-38b6-488e-ac2e-917201efa1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:49.170744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b40f15-38b6-488e-ac2e-917201efa1ce None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:49.171046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b40f15-38b6-488e-ac2e-917201efa1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:49.171470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b40f15-38b6-488e-ac2e-917201efa1ce None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/users/9c8f433f2f1f4923859810dc0019845c/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:49.173400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b40f15-38b6-488e-ac2e-917201efa1ce None admin] RBAC: Authorizing `identity:create_grant(project_id=cdbff3dc80ab4433ba8d78b52eb66086, user_id=9c8f433f2f1f4923859810dc0019845c, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:49.179019 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b40f15-38b6-488e-ac2e-917201efa1ce None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:49.193337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 65/131] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:47:49 2022] PUT /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/users/9c8f433f2f1f4923859810dc0019845c/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:47:49.194521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b40f15-38b6-488e-ac2e-917201efa1ce None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:49.344281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:49.386469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b40f15-38b6-488e-ac2e-917201efa1ce None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:49.452033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66c785-e423-485c-988b-487fa66bef0a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:49.458985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa6573-9354-446f-9baf-09a3583efd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:49.459673 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6573-9354-446f-9baf-09a3583efd54 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:49.459957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6573-9354-446f-9baf-09a3583efd54 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:49.460236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6573-9354-446f-9baf-09a3583efd54 None admin] PATH_INFO: `/v3/users/216c0adc13b34a498d52b72c7b75c72b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:49.463576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa6573-9354-446f-9baf-09a3583efd54 None admin] RBAC: Authorizing `identity:get_user(user_id=216c0adc13b34a498d52b72c7b75c72b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:49.465495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa6573-9354-446f-9baf-09a3583efd54 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:49.467829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 66/132] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:49 2022] GET /identity/v3/users/216c0adc13b34a498d52b72c7b75c72b => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:49.497036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa6573-9354-446f-9baf-09a3583efd54 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:49.512994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e40d25-a7b0-4132-9472-6a02067e232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:49.514961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e40d25-a7b0-4132-9472-6a02067e232c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:49.516544 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e40d25-a7b0-4132-9472-6a02067e232c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:49.516836 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e40d25-a7b0-4132-9472-6a02067e232c None admin] PATH_INFO: `/v3/users/216c0adc13b34a498d52b72c7b75c72b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:49.519684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e40d25-a7b0-4132-9472-6a02067e232c None admin] RBAC: Authorizing `identity:get_user(user_id=216c0adc13b34a498d52b72c7b75c72b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:49.521525 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e40d25-a7b0-4132-9472-6a02067e232c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:49.531583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 67/133] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:49 2022] GET /identity/v3/users/216c0adc13b34a498d52b72c7b75c72b => generated 275 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:49.535065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e40d25-a7b0-4132-9472-6a02067e232c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:49.557051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:49.691835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa6573-9354-446f-9baf-09a3583efd54 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:49.718459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e40d25-a7b0-4132-9472-6a02067e232c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:49.753647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:49.770514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8e0cb-b081-4713-a26e-b7c2bde6b055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:49.772533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8e0cb-b081-4713-a26e-b7c2bde6b055 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:49.773126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8e0cb-b081-4713-a26e-b7c2bde6b055 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:49.774028 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8e0cb-b081-4713-a26e-b7c2bde6b055 None admin] PATH_INFO: `/v3/projects/e8eb80afcc8c40d6bfae07f43cecacc8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:49.776777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8e0cb-b081-4713-a26e-b7c2bde6b055 None admin] RBAC: Authorizing `identity:get_project(project_id=e8eb80afcc8c40d6bfae07f43cecacc8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:49.780018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8e0cb-b081-4713-a26e-b7c2bde6b055 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:49.782275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 68/134] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:49 2022] GET /identity/v3/projects/e8eb80afcc8c40d6bfae07f43cecacc8 => generated 312 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:49.783660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3069d-c9dd-4385-93d8-a1de90596edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:49.784541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3069d-c9dd-4385-93d8-a1de90596edc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:49.784966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3069d-c9dd-4385-93d8-a1de90596edc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:49.785382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3069d-c9dd-4385-93d8-a1de90596edc None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:49.787334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30bcbd-5ab4-4786-849d-db442b7b9ccd None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:49.788124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3069d-c9dd-4385-93d8-a1de90596edc None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:49.789940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3069d-c9dd-4385-93d8-a1de90596edc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:49.791795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 67/135] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:49 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:49.792256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30bcbd-5ab4-4786-849d-db442b7b9ccd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:49.792703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30bcbd-5ab4-4786-849d-db442b7b9ccd None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:49.794245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 69/136] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:49 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:49.795176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3069d-c9dd-4385-93d8-a1de90596edc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:49.798665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30bcbd-5ab4-4786-849d-db442b7b9ccd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:49.980621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:49.986307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30bcbd-5ab4-4786-849d-db442b7b9ccd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:49.988368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3069d-c9dd-4385-93d8-a1de90596edc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:50.043824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7584c-ecf2-4793-81a4-9e32d50d1163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:50.045304 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7584c-ecf2-4793-81a4-9e32d50d1163 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:50.045544 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9992410-cd72-4357-8020-43bcf7efb69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:50.047296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9992410-cd72-4357-8020-43bcf7efb69b None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:50.048145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7584c-ecf2-4793-81a4-9e32d50d1163 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:50.048565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7584c-ecf2-4793-81a4-9e32d50d1163 None admin] PATH_INFO: `/v3/projects/e8eb80afcc8c40d6bfae07f43cecacc8/users/216c0adc13b34a498d52b72c7b75c72b/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:50.049768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9992410-cd72-4357-8020-43bcf7efb69b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:50.050668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9992410-cd72-4357-8020-43bcf7efb69b None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/users/216c0adc13b34a498d52b72c7b75c72b/roles/31cc9f3a35594d40924277f4778c9f03` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:50.052079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7584c-ecf2-4793-81a4-9e32d50d1163 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8eb80afcc8c40d6bfae07f43cecacc8, user_id=216c0adc13b34a498d52b72c7b75c72b, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:50.054709 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7584c-ecf2-4793-81a4-9e32d50d1163 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:50.055536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9992410-cd72-4357-8020-43bcf7efb69b None admin] RBAC: Authorizing `identity:create_grant(project_id=cdbff3dc80ab4433ba8d78b52eb66086, user_id=216c0adc13b34a498d52b72c7b75c72b, role_id=31cc9f3a35594d40924277f4778c9f03)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:50.057987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9992410-cd72-4357-8020-43bcf7efb69b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:50.082803 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 68/137] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:47:49 2022] PUT /identity/v3/projects/e8eb80afcc8c40d6bfae07f43cecacc8/users/216c0adc13b34a498d52b72c7b75c72b/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:47:50.084967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca5c5-e416-4b9f-aa58-6eb238d5c1ce None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:50.085855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 70/138] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:47:49 2022] PUT /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/users/216c0adc13b34a498d52b72c7b75c72b/roles/31cc9f3a35594d40924277f4778c9f03 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:47:50.086684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca5c5-e416-4b9f-aa58-6eb238d5c1ce None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:50.087346 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca5c5-e416-4b9f-aa58-6eb238d5c1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:50.112565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fdbca5c5-e416-4b9f-aa58-6eb238d5c1ce None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:50.114110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fdbca5c5-e416-4b9f-aa58-6eb238d5c1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:50.197315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 69/139] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:50.510759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3178807-c856-4922-b571-5dd0b8deb143 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:50.511452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3178807-c856-4922-b571-5dd0b8deb143 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:50.511940 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3178807-c856-4922-b571-5dd0b8deb143 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:50.514136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 71/140] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:50.520145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac88b33-3c51-4385-a6c4-534f23552419 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:50.520698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac88b33-3c51-4385-a6c4-534f23552419 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:50.521192 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac88b33-3c51-4385-a6c4-534f23552419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:50.547573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6ac88b33-3c51-4385-a6c4-534f23552419 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:50.548932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6ac88b33-3c51-4385-a6c4-534f23552419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:50.639907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3178807-c856-4922-b571-5dd0b8deb143 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:50.641649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 70/141] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:50.732404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:50.816801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3178807-c856-4922-b571-5dd0b8deb143 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:50.880766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba5864-570b-421d-a7c2-a179e8f6df03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:50.881434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5864-570b-421d-a7c2-a179e8f6df03 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:50.881720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5864-570b-421d-a7c2-a179e8f6df03 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:50.882002 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5864-570b-421d-a7c2-a179e8f6df03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:50.883181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba5864-570b-421d-a7c2-a179e8f6df03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:50.884651 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba5864-570b-421d-a7c2-a179e8f6df03 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:50.886592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 72/142] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:50.892386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac88b33-3c51-4385-a6c4-534f23552419 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:50.944433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:51.049846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba5864-570b-421d-a7c2-a179e8f6df03 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:51.098695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac88b33-3c51-4385-a6c4-534f23552419 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:51.135918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602ad64-596a-43b1-a4ad-518adbd66b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:51.136585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602ad64-596a-43b1-a4ad-518adbd66b9b None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:51.136865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602ad64-596a-43b1-a4ad-518adbd66b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:51.137134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602ad64-596a-43b1-a4ad-518adbd66b9b None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:51.137840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602ad64-596a-43b1-a4ad-518adbd66b9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:51.139430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602ad64-596a-43b1-a4ad-518adbd66b9b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:51.165003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 71/143] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:47:50 2022] POST /identity/v3/users => generated 283 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:47:51.217377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712f701-f8f8-4c9c-ad07-9c7ba6919eae None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:51.217935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712f701-f8f8-4c9c-ad07-9c7ba6919eae None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:51.218668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712f701-f8f8-4c9c-ad07-9c7ba6919eae None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:51.221450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 72/144] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:47:51.237183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35cef-a7d4-412c-b740-64db5b85d315 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:51.237476 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35cef-a7d4-412c-b740-64db5b85d315 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:51.237728 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35cef-a7d4-412c-b740-64db5b85d315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:51.271585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5be35cef-a7d4-412c-b740-64db5b85d315 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:51.272975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5be35cef-a7d4-412c-b740-64db5b85d315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:51.287771 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba5864-570b-421d-a7c2-a179e8f6df03 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:51.353974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:51.380479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3170d-60ba-4748-815c-9a9297b0cad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:51.381157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3170d-60ba-4748-815c-9a9297b0cad9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:51.381442 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3170d-60ba-4748-815c-9a9297b0cad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:51.381717 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3170d-60ba-4748-815c-9a9297b0cad9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:51.398333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3170d-60ba-4748-815c-9a9297b0cad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ceb3170d-60ba-4748-815c-9a9297b0cad9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:51.403877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:51.404873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:51.405688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 73/145] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:51.411194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3170d-60ba-4748-815c-9a9297b0cad9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:51.432635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 73/146] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:51.518631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3170d-60ba-4748-815c-9a9297b0cad9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:51.556075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6d3ef-3e3d-4e59-9e40-c21b83afa1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:51.556698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6d3ef-3e3d-4e59-9e40-c21b83afa1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:51.556982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6d3ef-3e3d-4e59-9e40-c21b83afa1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:51.557261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6d3ef-3e3d-4e59-9e40-c21b83afa1f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:51.557935 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6d3ef-3e3d-4e59-9e40-c21b83afa1f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:51.558242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6d3ef-3e3d-4e59-9e40-c21b83afa1f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:51.559654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6d3ef-3e3d-4e59-9e40-c21b83afa1f2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:51.570601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 74/147] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:47:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:51.576318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35cef-a7d4-412c-b740-64db5b85d315 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:51.595913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:51.686370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35cef-a7d4-412c-b740-64db5b85d315 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:51.726109 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1407a-8206-4a53-8df7-6945f501a34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:51.726777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1407a-8206-4a53-8df7-6945f501a34f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:51.727082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1407a-8206-4a53-8df7-6945f501a34f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:51.727361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1407a-8206-4a53-8df7-6945f501a34f None admin] PATH_INFO: `/v3/users/216c0adc13b34a498d52b72c7b75c72b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:51.728559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1407a-8206-4a53-8df7-6945f501a34f None admin] RBAC: Authorizing `identity:get_user(user_id=216c0adc13b34a498d52b72c7b75c72b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:51.730023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1407a-8206-4a53-8df7-6945f501a34f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:51.731820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 74/148] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:51 2022] GET /identity/v3/users/216c0adc13b34a498d52b72c7b75c72b => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:51.737703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6d3ef-3e3d-4e59-9e40-c21b83afa1f2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:51.791402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:51.860250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6d3ef-3e3d-4e59-9e40-c21b83afa1f2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:51.883152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1407a-8206-4a53-8df7-6945f501a34f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:51.907374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fd18c-307d-4c70-981f-0c3ce256cea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:51.907569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fd18c-307d-4c70-981f-0c3ce256cea1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:51.907813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fd18c-307d-4c70-981f-0c3ce256cea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:51.908257 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fd18c-307d-4c70-981f-0c3ce256cea1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:51.991373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:51.994479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fd18c-307d-4c70-981f-0c3ce256cea1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:51.996177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fd18c-307d-4c70-981f-0c3ce256cea1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:52.049777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:52.061475 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1407a-8206-4a53-8df7-6945f501a34f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:52.115741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 75/149] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:47:52.117245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad33178-fe37-4874-813c-f68e8452c8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:52.118029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad33178-fe37-4874-813c-f68e8452c8dd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:52.118381 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad33178-fe37-4874-813c-f68e8452c8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:52.118782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad33178-fe37-4874-813c-f68e8452c8dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:52.124151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fd18c-307d-4c70-981f-0c3ce256cea1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:52.141118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad33178-fe37-4874-813c-f68e8452c8dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fad33178-fe37-4874-813c-f68e8452c8dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:52.146515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:52.151626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:47:52.152394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:47:52.154688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 75/150] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:47:52.161119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad33178-fe37-4874-813c-f68e8452c8dd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:52.267161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fd18c-307d-4c70-981f-0c3ce256cea1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:52.292056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad33178-fe37-4874-813c-f68e8452c8dd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:52.308840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f35cc-b37a-4554-b731-209a09e13ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:52.309840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f35cc-b37a-4554-b731-209a09e13ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:52.310281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f35cc-b37a-4554-b731-209a09e13ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:52.310729 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f35cc-b37a-4554-b731-209a09e13ac5 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:52.312649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f35cc-b37a-4554-b731-209a09e13ac5 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:52.314695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f35cc-b37a-4554-b731-209a09e13ac5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:52.316567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 76/151] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:52 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:52.317856 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec461-e79e-465b-b592-48e9d266f7b7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:52.318285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec461-e79e-465b-b592-48e9d266f7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:52.319041 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec461-e79e-465b-b592-48e9d266f7b7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:52.320492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 77/152] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:52.322664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eec461-e79e-465b-b592-48e9d266f7b7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:52.330927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:52.339140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d3b20-648a-4ef7-a46a-98705bba186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:52.339882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d3b20-648a-4ef7-a46a-98705bba186d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:52.340218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d3b20-648a-4ef7-a46a-98705bba186d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:52.340567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d3b20-648a-4ef7-a46a-98705bba186d None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:52.341355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d3b20-648a-4ef7-a46a-98705bba186d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:52.341769 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d3b20-648a-4ef7-a46a-98705bba186d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:52.343348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d3b20-648a-4ef7-a46a-98705bba186d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:52.346635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:52.354510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 76/153] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:47:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:52.355904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c691a-e021-4380-8e8f-c0e46e30c636 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:52.356269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c691a-e021-4380-8e8f-c0e46e30c636 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:52.357003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c691a-e021-4380-8e8f-c0e46e30c636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:52.378512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-366c691a-e021-4380-8e8f-c0e46e30c636 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:52.379921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-366c691a-e021-4380-8e8f-c0e46e30c636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:52.467218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eec461-e79e-465b-b592-48e9d266f7b7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:52.469357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 77/154] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:52.470457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c691a-e021-4380-8e8f-c0e46e30c636 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:52.544803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5715365-726d-44ed-9d50-5f2bf3ac8e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:52.545411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5715365-726d-44ed-9d50-5f2bf3ac8e7e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:52.545763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5715365-726d-44ed-9d50-5f2bf3ac8e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:52.545970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5715365-726d-44ed-9d50-5f2bf3ac8e7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:52.546757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5715365-726d-44ed-9d50-5f2bf3ac8e7e None admin] RBAC: Adding query filter params (role.id=a0db5bd603fc4096917bbcd8c6932bc0, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086, user.id=216c0adc13b34a498d52b72c7b75c72b) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:52.547094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5715365-726d-44ed-9d50-5f2bf3ac8e7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:52.548924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5715365-726d-44ed-9d50-5f2bf3ac8e7e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:52.563351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 78/155] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:52 2022] GET /identity/v3/role_assignments?user.id=216c0adc13b34a498d52b72c7b75c72b&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:52.594683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c691a-e021-4380-8e8f-c0e46e30c636 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:52.622845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:52.636752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522865a7-e1f8-4770-bca1-8bda85d57b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:52.637412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522865a7-e1f8-4770-bca1-8bda85d57b61 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:52.637843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522865a7-e1f8-4770-bca1-8bda85d57b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:52.638347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522865a7-e1f8-4770-bca1-8bda85d57b61 None admin] PATH_INFO: `/v3/users/fbfa2ddf17434fa09745d5a1fa32cb4f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:52.640664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522865a7-e1f8-4770-bca1-8bda85d57b61 None admin] RBAC: Authorizing `identity:get_user(user_id=fbfa2ddf17434fa09745d5a1fa32cb4f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:52.642241 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522865a7-e1f8-4770-bca1-8bda85d57b61 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:52.643989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 78/156] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:52 2022] GET /identity/v3/users/fbfa2ddf17434fa09745d5a1fa32cb4f => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:52.649912 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5715365-726d-44ed-9d50-5f2bf3ac8e7e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:52.763852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:52.790486 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5715365-726d-44ed-9d50-5f2bf3ac8e7e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:52.835818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf15d7-c3d2-468c-8b72-f35975c01e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:52.836395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf15d7-c3d2-468c-8b72-f35975c01e52 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:52.836678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf15d7-c3d2-468c-8b72-f35975c01e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:52.836954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf15d7-c3d2-468c-8b72-f35975c01e52 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:52.838191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf15d7-c3d2-468c-8b72-f35975c01e52 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:52.839805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf15d7-c3d2-468c-8b72-f35975c01e52 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:52.841719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 79/157] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:52 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:52.847480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522865a7-e1f8-4770-bca1-8bda85d57b61 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:52.928663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf15d7-c3d2-468c-8b72-f35975c01e52 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:53.019024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522865a7-e1f8-4770-bca1-8bda85d57b61 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:53.065860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf15d7-c3d2-468c-8b72-f35975c01e52 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:53.106761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223e09b-b25a-4cbf-bc4c-b955a67c2e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:53.106940 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24914036-56d4-41ab-b262-e96ead387a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:53.107389 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223e09b-b25a-4cbf-bc4c-b955a67c2e80 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:53.107673 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223e09b-b25a-4cbf-bc4c-b955a67c2e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:53.107947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223e09b-b25a-4cbf-bc4c-b955a67c2e80 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/users/fbfa2ddf17434fa09745d5a1fa32cb4f/roles/4b69aa55fbbf4624b0b6036a5a8b2d4d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:53.108473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24914036-56d4-41ab-b262-e96ead387a2a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:53.109767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24914036-56d4-41ab-b262-e96ead387a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:53.110373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24914036-56d4-41ab-b262-e96ead387a2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:53.115857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223e09b-b25a-4cbf-bc4c-b955a67c2e80 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdbff3dc80ab4433ba8d78b52eb66086, user_id=fbfa2ddf17434fa09745d5a1fa32cb4f, role_id=4b69aa55fbbf4624b0b6036a5a8b2d4d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:53.117337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223e09b-b25a-4cbf-bc4c-b955a67c2e80 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:53.124686 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24914036-56d4-41ab-b262-e96ead387a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-24914036-56d4-41ab-b262-e96ead387a2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:53.132195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:53.133161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:53.134034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:53.136636 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 80/158] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:47:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:47:53.141249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 79/159] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:47:52 2022] PUT /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/users/fbfa2ddf17434fa09745d5a1fa32cb4f/roles/4b69aa55fbbf4624b0b6036a5a8b2d4d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:47:53.142445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24914036-56d4-41ab-b262-e96ead387a2a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:53.152495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:53.280182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24914036-56d4-41ab-b262-e96ead387a2a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:53.330177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f62ca-8965-4938-b275-73e0eab4da51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:53.331210 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f62ca-8965-4938-b275-73e0eab4da51 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:53.331734 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f62ca-8965-4938-b275-73e0eab4da51 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:53.332227 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f62ca-8965-4938-b275-73e0eab4da51 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:53.333140 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f62ca-8965-4938-b275-73e0eab4da51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:53.333642 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f62ca-8965-4938-b275-73e0eab4da51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:53.335300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f62ca-8965-4938-b275-73e0eab4da51 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:53.345681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 81/160] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:53.356146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223e09b-b25a-4cbf-bc4c-b955a67c2e80 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:53.434369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c90074-ec08-42c6-ae47-551ee88b062e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:53.435045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c90074-ec08-42c6-ae47-551ee88b062e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:53.435584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c90074-ec08-42c6-ae47-551ee88b062e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:53.437200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 82/161] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:47:53.442485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142a64e-38f7-45c7-8c89-99d40c6e7ff8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:53.443124 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142a64e-38f7-45c7-8c89-99d40c6e7ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:53.443866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142a64e-38f7-45c7-8c89-99d40c6e7ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:53.444528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:53.480003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d142a64e-38f7-45c7-8c89-99d40c6e7ff8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:53.481501 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d142a64e-38f7-45c7-8c89-99d40c6e7ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:53.503339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223e09b-b25a-4cbf-bc4c-b955a67c2e80 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:53.555723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f931f-defb-4820-ba97-0da127189ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:53.556578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f931f-defb-4820-ba97-0da127189ede None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:53.556994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f931f-defb-4820-ba97-0da127189ede None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:53.557278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f931f-defb-4820-ba97-0da127189ede None admin] PATH_INFO: `/v3/users/9c8f433f2f1f4923859810dc0019845c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:53.559372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f931f-defb-4820-ba97-0da127189ede None admin] RBAC: Authorizing `identity:get_user(user_id=9c8f433f2f1f4923859810dc0019845c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:53.561017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f931f-defb-4820-ba97-0da127189ede None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:53.563023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 80/162] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:53 2022] GET /identity/v3/users/9c8f433f2f1f4923859810dc0019845c => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:53.566187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:53.568511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f931f-defb-4820-ba97-0da127189ede None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:53.572889 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 83/163] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:53.690086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f931f-defb-4820-ba97-0da127189ede None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:53.726159 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:53.744119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7586c20-5f19-4ca5-b580-6dfe27b8c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:53.745144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7586c20-5f19-4ca5-b580-6dfe27b8c3cf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:53.745486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7586c20-5f19-4ca5-b580-6dfe27b8c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:53.745821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7586c20-5f19-4ca5-b580-6dfe27b8c3cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:53.821695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8865d6a-bbd9-496b-9745-41413343af43 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:53.821998 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8865d6a-bbd9-496b-9745-41413343af43 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:53.822296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8865d6a-bbd9-496b-9745-41413343af43 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:53.824039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 84/164] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:53.831167 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53afa0b-a432-46d2-bb96-811f827a47c8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:53.831448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53afa0b-a432-46d2-bb96-811f827a47c8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:53.831719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53afa0b-a432-46d2-bb96-811f827a47c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:53.857149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c53afa0b-a432-46d2-bb96-811f827a47c8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:53.858337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c53afa0b-a432-46d2-bb96-811f827a47c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:53.863960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7586c20-5f19-4ca5-b580-6dfe27b8c3cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:53.865506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7586c20-5f19-4ca5-b580-6dfe27b8c3cf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:53.999170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 85/165] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:54.030577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53afa0b-a432-46d2-bb96-811f827a47c8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:54.058955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 81/166] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:47:54.065266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7586c20-5f19-4ca5-b580-6dfe27b8c3cf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:54.106309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:54.234684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7586c20-5f19-4ca5-b580-6dfe27b8c3cf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:54.261265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53afa0b-a432-46d2-bb96-811f827a47c8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:54.300738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a22fea-510d-406f-b75c-57ee3f527ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:54.301504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a22fea-510d-406f-b75c-57ee3f527ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:54.301809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a22fea-510d-406f-b75c-57ee3f527ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:54.302053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a22fea-510d-406f-b75c-57ee3f527ef1 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:54.303391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a22fea-510d-406f-b75c-57ee3f527ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:54.305279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a22fea-510d-406f-b75c-57ee3f527ef1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:54.306986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 82/167] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:54 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:54.315980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a22fea-510d-406f-b75c-57ee3f527ef1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:54.316502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:54.332668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c3d42-de94-4bdc-b222-a051dc3e7026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:54.333427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c3d42-de94-4bdc-b222-a051dc3e7026 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:54.333712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c3d42-de94-4bdc-b222-a051dc3e7026 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:54.333981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c3d42-de94-4bdc-b222-a051dc3e7026 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:54.359986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c3d42-de94-4bdc-b222-a051dc3e7026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ef4c3d42-de94-4bdc-b222-a051dc3e7026 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:54.365304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:54.366385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:54.367597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 86/168] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:54.376071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c3d42-de94-4bdc-b222-a051dc3e7026 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:54.507641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:54.535956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a22fea-510d-406f-b75c-57ee3f527ef1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:54.581493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c67fda-9c73-4f4a-99ec-83c8924dcc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:54.582377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c67fda-9c73-4f4a-99ec-83c8924dcc49 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:54.582792 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c67fda-9c73-4f4a-99ec-83c8924dcc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:54.583225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c67fda-9c73-4f4a-99ec-83c8924dcc49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:54.584699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c67fda-9c73-4f4a-99ec-83c8924dcc49 None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086, user.id=9c8f433f2f1f4923859810dc0019845c) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:54.585162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c67fda-9c73-4f4a-99ec-83c8924dcc49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:54.586751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c67fda-9c73-4f4a-99ec-83c8924dcc49 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:54.595026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 83/169] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:54 2022] GET /identity/v3/role_assignments?user.id=9c8f433f2f1f4923859810dc0019845c&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:54.596423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c67fda-9c73-4f4a-99ec-83c8924dcc49 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:54.602863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c3d42-de94-4bdc-b222-a051dc3e7026 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:54.699778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df2e73-4169-4129-a8e6-314393e0b85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:54.700412 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df2e73-4169-4129-a8e6-314393e0b85d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:54.700689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df2e73-4169-4129-a8e6-314393e0b85d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:54.700978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df2e73-4169-4129-a8e6-314393e0b85d None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:54.701651 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df2e73-4169-4129-a8e6-314393e0b85d None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:54.701959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df2e73-4169-4129-a8e6-314393e0b85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:54.703440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df2e73-4169-4129-a8e6-314393e0b85d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:54.724785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 87/170] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:47:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:54.731173 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df2e73-4169-4129-a8e6-314393e0b85d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:54.764178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c67fda-9c73-4f4a-99ec-83c8924dcc49 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:54.831862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb701d4e-bfe6-487f-8a85-9a232f5c14ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:54.832852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb701d4e-bfe6-487f-8a85-9a232f5c14ab None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:54.833371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb701d4e-bfe6-487f-8a85-9a232f5c14ab None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:54.833838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb701d4e-bfe6-487f-8a85-9a232f5c14ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:54.855157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb701d4e-bfe6-487f-8a85-9a232f5c14ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-eb701d4e-bfe6-487f-8a85-9a232f5c14ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:54.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:54.861078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:47:54.861952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 84/171] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:47:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:54.869004 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb701d4e-bfe6-487f-8a85-9a232f5c14ab None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:54.874517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:54.900885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:54.973201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df2e73-4169-4129-a8e6-314393e0b85d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:55.028852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb701d4e-bfe6-487f-8a85-9a232f5c14ab None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:55.031882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c261f1-68a4-45b0-9552-6a2d8009661f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:55.032539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c261f1-68a4-45b0-9552-6a2d8009661f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:55.032825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c261f1-68a4-45b0-9552-6a2d8009661f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:55.033234 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c261f1-68a4-45b0-9552-6a2d8009661f None admin] PATH_INFO: `/v3/users/216c0adc13b34a498d52b72c7b75c72b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:55.035740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c261f1-68a4-45b0-9552-6a2d8009661f None admin] RBAC: Authorizing `identity:get_user(user_id=216c0adc13b34a498d52b72c7b75c72b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:55.037499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c261f1-68a4-45b0-9552-6a2d8009661f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:55.039453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 88/172] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:54 2022] GET /identity/v3/users/216c0adc13b34a498d52b72c7b75c72b => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:55.048581 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c261f1-68a4-45b0-9552-6a2d8009661f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:55.088232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a5a5c-0bd2-42f4-b2fe-3171dae6fe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:55.088795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a5a5c-0bd2-42f4-b2fe-3171dae6fe34 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:55.089077 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a5a5c-0bd2-42f4-b2fe-3171dae6fe34 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:55.089351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a5a5c-0bd2-42f4-b2fe-3171dae6fe34 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:55.090038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a5a5c-0bd2-42f4-b2fe-3171dae6fe34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:55.090345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a5a5c-0bd2-42f4-b2fe-3171dae6fe34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:55.091925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a5a5c-0bd2-42f4-b2fe-3171dae6fe34 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:55.107962 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 85/173] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:47:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:55.113022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a5a5c-0bd2-42f4-b2fe-3171dae6fe34 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:55.207903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c261f1-68a4-45b0-9552-6a2d8009661f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:55.228592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:55.235392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:55.248508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd0038-cd28-46ff-9522-3edb7e8bf8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:55.249995 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd0038-cd28-46ff-9522-3edb7e8bf8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:55.250476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd0038-cd28-46ff-9522-3edb7e8bf8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:55.251093 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd0038-cd28-46ff-9522-3edb7e8bf8c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:55.272429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a5a5c-0bd2-42f4-b2fe-3171dae6fe34 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:55.333418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd0038-cd28-46ff-9522-3edb7e8bf8c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:55.335579 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd0038-cd28-46ff-9522-3edb7e8bf8c0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:55.335777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69333a4-786b-4c11-8c83-6233b9cc8d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:55.336386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69333a4-786b-4c11-8c83-6233b9cc8d42 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:55.336673 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69333a4-786b-4c11-8c83-6233b9cc8d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:55.336955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69333a4-786b-4c11-8c83-6233b9cc8d42 None admin] PATH_INFO: `/v3/users/216c0adc13b34a498d52b72c7b75c72b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:55.338166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69333a4-786b-4c11-8c83-6233b9cc8d42 None admin] RBAC: Authorizing `identity:get_user(user_id=216c0adc13b34a498d52b72c7b75c72b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:55.339693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69333a4-786b-4c11-8c83-6233b9cc8d42 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:55.341414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 86/174] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:55 2022] GET /identity/v3/users/216c0adc13b34a498d52b72c7b75c72b => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:55.347046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69333a4-786b-4c11-8c83-6233b9cc8d42 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:55.511051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69333a4-786b-4c11-8c83-6233b9cc8d42 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:55.542659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 89/175] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:47:55.543489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd0038-cd28-46ff-9522-3edb7e8bf8c0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:55.574787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9309688-b51b-48b9-ad89-97c71e2de5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:55.575487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9309688-b51b-48b9-ad89-97c71e2de5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:55.575807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9309688-b51b-48b9-ad89-97c71e2de5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:55.576106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9309688-b51b-48b9-ad89-97c71e2de5b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:55.582720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:55.601366 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:55.696699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9309688-b51b-48b9-ad89-97c71e2de5b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:55.698765 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9309688-b51b-48b9-ad89-97c71e2de5b1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:55.781037 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd0038-cd28-46ff-9522-3edb7e8bf8c0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:55.822881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de9313-52c9-40f4-b0a9-c830363e0d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:55.823554 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de9313-52c9-40f4-b0a9-c830363e0d86 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:55.823834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de9313-52c9-40f4-b0a9-c830363e0d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:55.824108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de9313-52c9-40f4-b0a9-c830363e0d86 None admin] PATH_INFO: `/v3/projects/e8eb80afcc8c40d6bfae07f43cecacc8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:55.825371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de9313-52c9-40f4-b0a9-c830363e0d86 None admin] RBAC: Authorizing `identity:get_project(project_id=e8eb80afcc8c40d6bfae07f43cecacc8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:55.826823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de9313-52c9-40f4-b0a9-c830363e0d86 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:55.830743 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 90/176] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:55 2022] GET /identity/v3/projects/e8eb80afcc8c40d6bfae07f43cecacc8 => generated 312 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:55.831972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c89e4-8bf7-42d5-a34c-2733d9940edf None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:55.832253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c89e4-8bf7-42d5-a34c-2733d9940edf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:55.832526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c89e4-8bf7-42d5-a34c-2733d9940edf None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:55.833771 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 91/177] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:55.840537 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214ba2b-2a51-4c9a-8b3a-944eb7877e24 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:55.840829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214ba2b-2a51-4c9a-8b3a-944eb7877e24 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:55.841108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214ba2b-2a51-4c9a-8b3a-944eb7877e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:55.843200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 87/178] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:47:55.844407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9309688-b51b-48b9-ad89-97c71e2de5b1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:55.880567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e214ba2b-2a51-4c9a-8b3a-944eb7877e24 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:55.883265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e214ba2b-2a51-4c9a-8b3a-944eb7877e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:55.919372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:55.974690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9309688-b51b-48b9-ad89-97c71e2de5b1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:56.001547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 92/179] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:56.002669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214ba2b-2a51-4c9a-8b3a-944eb7877e24 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:56.015546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e073403-d2ac-401a-93c7-adfc6b8c8224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:56.016455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e073403-d2ac-401a-93c7-adfc6b8c8224 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.016943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e073403-d2ac-401a-93c7-adfc6b8c8224 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.017477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e073403-d2ac-401a-93c7-adfc6b8c8224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.018534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e073403-d2ac-401a-93c7-adfc6b8c8224 None admin] RBAC: Adding query filter params (role.id=a0db5bd603fc4096917bbcd8c6932bc0, scope.project.id=e8eb80afcc8c40d6bfae07f43cecacc8, user.id=216c0adc13b34a498d52b72c7b75c72b) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:56.019193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e073403-d2ac-401a-93c7-adfc6b8c8224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:56.020842 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e073403-d2ac-401a-93c7-adfc6b8c8224 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:56.021646 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:56.033239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 88/180] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:55 2022] GET /identity/v3/role_assignments?user.id=216c0adc13b34a498d52b72c7b75c72b&scope.project.id=e8eb80afcc8c40d6bfae07f43cecacc8&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:56.039761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d3228-6b8b-47c7-92d6-4fb502a3c8a4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.040344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d3228-6b8b-47c7-92d6-4fb502a3c8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.040828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d3228-6b8b-47c7-92d6-4fb502a3c8a4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.042553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 89/181] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:56.050135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93db74-f85d-4ba5-b5f6-229747af0873 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.050764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93db74-f85d-4ba5-b5f6-229747af0873 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.051273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93db74-f85d-4ba5-b5f6-229747af0873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.080398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4c93db74-f85d-4ba5-b5f6-229747af0873 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:56.082010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4c93db74-f85d-4ba5-b5f6-229747af0873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:56.137430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214ba2b-2a51-4c9a-8b3a-944eb7877e24 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:56.203829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2bbf7-9116-49e7-9d19-80be8a067d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:56.205660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2bbf7-9116-49e7-9d19-80be8a067d63 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.206359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2bbf7-9116-49e7-9d19-80be8a067d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.208482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2bbf7-9116-49e7-9d19-80be8a067d63 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.210100 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2bbf7-9116-49e7-9d19-80be8a067d63 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:56.213487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2bbf7-9116-49e7-9d19-80be8a067d63 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:56.216427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 93/182] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:56 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:56.217254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b5982-439e-493f-9d27-e06fe1b4fc28 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.217541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b5982-439e-493f-9d27-e06fe1b4fc28 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.217815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b5982-439e-493f-9d27-e06fe1b4fc28 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.219684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 94/183] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:56.220589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 90/184] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:56.225154 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3f30a-be9e-43cd-9a95-b5b943c634ac None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.225440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3f30a-be9e-43cd-9a95-b5b943c634ac None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.225710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3f30a-be9e-43cd-9a95-b5b943c634ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.228481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93db74-f85d-4ba5-b5f6-229747af0873 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:56.249886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8cb3f30a-be9e-43cd-9a95-b5b943c634ac None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:56.251609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8cb3f30a-be9e-43cd-9a95-b5b943c634ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:56.269870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:56.374383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 95/185] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:56.395652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93db74-f85d-4ba5-b5f6-229747af0873 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:56.427061 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3f30a-be9e-43cd-9a95-b5b943c634ac None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:56.440969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cbcc0-4740-4933-88be-fe7674e3df0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:56.441946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cbcc0-4740-4933-88be-fe7674e3df0d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.442427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cbcc0-4740-4933-88be-fe7674e3df0d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.442914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cbcc0-4740-4933-88be-fe7674e3df0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.444124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cbcc0-4740-4933-88be-fe7674e3df0d None admin] RBAC: Adding query filter params (role.id=31cc9f3a35594d40924277f4778c9f03, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086, user.id=216c0adc13b34a498d52b72c7b75c72b) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:56.444628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cbcc0-4740-4933-88be-fe7674e3df0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:56.446339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cbcc0-4740-4933-88be-fe7674e3df0d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:56.453010 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:56.453708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 91/186] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:56 2022] GET /identity/v3/role_assignments?user.id=216c0adc13b34a498d52b72c7b75c72b&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=31cc9f3a35594d40924277f4778c9f03 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:56.517339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa7ed0-a62b-490e-9da1-8cb0942fd231 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.517961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa7ed0-a62b-490e-9da1-8cb0942fd231 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.518439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa7ed0-a62b-490e-9da1-8cb0942fd231 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.520110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 92/187] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:47:56.526710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8c8e-3996-4941-b718-483e55f41fd2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.527298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8c8e-3996-4941-b718-483e55f41fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.527765 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8c8e-3996-4941-b718-483e55f41fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.552493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0c0b8c8e-3996-4941-b718-483e55f41fd2 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:56.554079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0c0b8c8e-3996-4941-b718-483e55f41fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:56.621476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3f30a-be9e-43cd-9a95-b5b943c634ac None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:56.640929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 93/188] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:56.642295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b8c8e-3996-4941-b718-483e55f41fd2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:56.669614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ecbb7-c02b-4db4-a504-f01f188a9ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:56.670577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ecbb7-c02b-4db4-a504-f01f188a9ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.671120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ecbb7-c02b-4db4-a504-f01f188a9ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.671611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ecbb7-c02b-4db4-a504-f01f188a9ea3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.692175 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ecbb7-c02b-4db4-a504-f01f188a9ea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:56.695352 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4d7ecbb7-c02b-4db4-a504-f01f188a9ea3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:56.698701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:56.699655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:56.700504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 96/189] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:47:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:47:56.706926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ecbb7-c02b-4db4-a504-f01f188a9ea3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:56.796529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b8c8e-3996-4941-b718-483e55f41fd2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:56.804383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:56.834357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ecbb7-c02b-4db4-a504-f01f188a9ea3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:56.835163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7282f0-8426-411a-b7c0-65f3d1567245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:56.835867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7282f0-8426-411a-b7c0-65f3d1567245 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.836203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7282f0-8426-411a-b7c0-65f3d1567245 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.836528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7282f0-8426-411a-b7c0-65f3d1567245 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.837650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7282f0-8426-411a-b7c0-65f3d1567245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:56.839428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7282f0-8426-411a-b7c0-65f3d1567245 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:56.840691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 94/190] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:56.841463 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7282f0-8426-411a-b7c0-65f3d1567245 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:56.880367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6b4c3-0f3f-4bf3-ab4a-c4a711259e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:56.880979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6b4c3-0f3f-4bf3-ab4a-c4a711259e5c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.881257 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6b4c3-0f3f-4bf3-ab4a-c4a711259e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.881531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6b4c3-0f3f-4bf3-ab4a-c4a711259e5c None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.882201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6b4c3-0f3f-4bf3-ab4a-c4a711259e5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:56.882510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6b4c3-0f3f-4bf3-ab4a-c4a711259e5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:56.884226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6b4c3-0f3f-4bf3-ab4a-c4a711259e5c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:56.894173 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 97/191] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:56.895294 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6b4c3-0f3f-4bf3-ab4a-c4a711259e5c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:56.955303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:56.958729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7282f0-8426-411a-b7c0-65f3d1567245 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:56.992656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cb167-f001-4e76-85d1-6a2f426dc4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:56.993292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cb167-f001-4e76-85d1-6a2f426dc4db None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:56.993569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cb167-f001-4e76-85d1-6a2f426dc4db None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:56.993837 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cb167-f001-4e76-85d1-6a2f426dc4db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:56.996018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6b4c3-0f3f-4bf3-ab4a-c4a711259e5c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:57.004417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cb167-f001-4e76-85d1-6a2f426dc4db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-533cb167-f001-4e76-85d1-6a2f426dc4db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:57.009264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:57.010211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 95/192] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:47:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:57.011301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cb167-f001-4e76-85d1-6a2f426dc4db None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:57.034186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c1ad5-e4f2-404e-af5f-0b80bec7f4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:57.034788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c1ad5-e4f2-404e-af5f-0b80bec7f4e5 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:57.035106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c1ad5-e4f2-404e-af5f-0b80bec7f4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:57.035380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c1ad5-e4f2-404e-af5f-0b80bec7f4e5 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:57.036020 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c1ad5-e4f2-404e-af5f-0b80bec7f4e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:57.037464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c1ad5-e4f2-404e-af5f-0b80bec7f4e5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:57.046341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:57.059034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 98/193] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:47:56 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:47:57.059834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c1ad5-e4f2-404e-af5f-0b80bec7f4e5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:57.125770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cb167-f001-4e76-85d1-6a2f426dc4db None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:57.163215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7434fa-c6c7-46bf-9aad-a02436a79e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:57.163867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7434fa-c6c7-46bf-9aad-a02436a79e2a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:57.164149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7434fa-c6c7-46bf-9aad-a02436a79e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:57.164424 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7434fa-c6c7-46bf-9aad-a02436a79e2a None admin] PATH_INFO: `/v3/users/31dfdfab58dc4039933640d2aa79310a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:57.175773 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c1ad5-e4f2-404e-af5f-0b80bec7f4e5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:57.179946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7434fa-c6c7-46bf-9aad-a02436a79e2a None admin] RBAC: Authorizing `identity:get_user(user_id=31dfdfab58dc4039933640d2aa79310a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:57.181461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7434fa-c6c7-46bf-9aad-a02436a79e2a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:57.188213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 96/194] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:57 2022] GET /identity/v3/users/31dfdfab58dc4039933640d2aa79310a => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:57.188213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7434fa-c6c7-46bf-9aad-a02436a79e2a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:57.205075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:57.226480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04251c0d-76ca-4192-b302-a7c74135f773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:57.227071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04251c0d-76ca-4192-b302-a7c74135f773 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:57.227354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04251c0d-76ca-4192-b302-a7c74135f773 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:57.227627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04251c0d-76ca-4192-b302-a7c74135f773 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:57.243893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04251c0d-76ca-4192-b302-a7c74135f773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-04251c0d-76ca-4192-b302-a7c74135f773 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:57.249463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:57.250349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:57.251198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 99/195] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:57.252554 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04251c0d-76ca-4192-b302-a7c74135f773 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:57.302438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7434fa-c6c7-46bf-9aad-a02436a79e2a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:57.317268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:57.348083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bececbeb-1c24-460a-858f-5b61902a463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:57.348972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bececbeb-1c24-460a-858f-5b61902a463d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:57.349400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bececbeb-1c24-460a-858f-5b61902a463d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:57.349807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bececbeb-1c24-460a-858f-5b61902a463d None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:57.350614 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bececbeb-1c24-460a-858f-5b61902a463d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:57.351637 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bececbeb-1c24-460a-858f-5b61902a463d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:57.353208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bececbeb-1c24-460a-858f-5b61902a463d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:57.364071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 97/196] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:47:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:57.365191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bececbeb-1c24-460a-858f-5b61902a463d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:57.385046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04251c0d-76ca-4192-b302-a7c74135f773 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:57.427384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a159b4-b807-417e-9745-655bc24a858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:57.428588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a159b4-b807-417e-9745-655bc24a858b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:57.431038 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a159b4-b807-417e-9745-655bc24a858b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:57.431038 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a159b4-b807-417e-9745-655bc24a858b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:57.471344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:57.493834 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bececbeb-1c24-460a-858f-5b61902a463d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:57.520755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a159b4-b807-417e-9745-655bc24a858b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:57.523040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a159b4-b807-417e-9745-655bc24a858b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:57.535676 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76ca1d-1608-4fd9-95fe-f9331cab3612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:57.536749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76ca1d-1608-4fd9-95fe-f9331cab3612 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:57.537036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76ca1d-1608-4fd9-95fe-f9331cab3612 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:57.537314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76ca1d-1608-4fd9-95fe-f9331cab3612 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:57.537981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76ca1d-1608-4fd9-95fe-f9331cab3612 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:57.538282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76ca1d-1608-4fd9-95fe-f9331cab3612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:57.540501 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76ca1d-1608-4fd9-95fe-f9331cab3612 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:57.552178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 98/197] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:47:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:57.553142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76ca1d-1608-4fd9-95fe-f9331cab3612 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:57.598946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:57.665560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 100/198] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:47:57.666798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a159b4-b807-417e-9745-655bc24a858b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:57.701223 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76ca1d-1608-4fd9-95fe-f9331cab3612 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:57.749043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773f637-0293-461a-88d8-1b49aa8b55ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:57.750425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773f637-0293-461a-88d8-1b49aa8b55ba None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:57.750705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773f637-0293-461a-88d8-1b49aa8b55ba None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:57.750979 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773f637-0293-461a-88d8-1b49aa8b55ba None admin] PATH_INFO: `/v3/users/31dfdfab58dc4039933640d2aa79310a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:57.752886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773f637-0293-461a-88d8-1b49aa8b55ba None admin] RBAC: Authorizing `identity:get_user(user_id=31dfdfab58dc4039933640d2aa79310a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:57.754629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773f637-0293-461a-88d8-1b49aa8b55ba None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:57.757456 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 99/199] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:57 2022] GET /identity/v3/users/31dfdfab58dc4039933640d2aa79310a => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:57.758295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773f637-0293-461a-88d8-1b49aa8b55ba None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:57.773051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:57.803698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a159b4-b807-417e-9745-655bc24a858b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:57.845223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117baa93-23c6-49bd-a4f4-03dc7dcc19e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:57.846378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117baa93-23c6-49bd-a4f4-03dc7dcc19e9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:57.846933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117baa93-23c6-49bd-a4f4-03dc7dcc19e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:57.847457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117baa93-23c6-49bd-a4f4-03dc7dcc19e9 None admin] PATH_INFO: `/v3/users/fbfa2ddf17434fa09745d5a1fa32cb4f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:57.848837 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117baa93-23c6-49bd-a4f4-03dc7dcc19e9 None admin] RBAC: Authorizing `identity:get_user(user_id=fbfa2ddf17434fa09745d5a1fa32cb4f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:57.850836 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117baa93-23c6-49bd-a4f4-03dc7dcc19e9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:57.852663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 101/200] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:47:57 2022] GET /identity/v3/users/fbfa2ddf17434fa09745d5a1fa32cb4f => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:57.853818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117baa93-23c6-49bd-a4f4-03dc7dcc19e9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:57.887096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:57.887653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773f637-0293-461a-88d8-1b49aa8b55ba None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:57.934149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15684e26-6576-4976-a8a8-80530956dd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:57.935029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15684e26-6576-4976-a8a8-80530956dd90 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:57.935321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15684e26-6576-4976-a8a8-80530956dd90 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:57.935596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15684e26-6576-4976-a8a8-80530956dd90 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:57.937313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15684e26-6576-4976-a8a8-80530956dd90 None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:57.939140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15684e26-6576-4976-a8a8-80530956dd90 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:57.946467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 100/201] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:57 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:57.946467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15684e26-6576-4976-a8a8-80530956dd90 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:57.981812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117baa93-23c6-49bd-a4f4-03dc7dcc19e9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:58.020313 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef46f3a-c14f-4cd7-9301-6122bdd4886a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:58.021326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef46f3a-c14f-4cd7-9301-6122bdd4886a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:58.021893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef46f3a-c14f-4cd7-9301-6122bdd4886a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:58.022362 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef46f3a-c14f-4cd7-9301-6122bdd4886a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:58.043593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:58.082805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15684e26-6576-4976-a8a8-80530956dd90 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:58.106550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef46f3a-c14f-4cd7-9301-6122bdd4886a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:58.108985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef46f3a-c14f-4cd7-9301-6122bdd4886a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:58.129957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d1785-0cad-4628-ba0e-32d3006d6de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:58.130587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d1785-0cad-4628-ba0e-32d3006d6de2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:58.130864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d1785-0cad-4628-ba0e-32d3006d6de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:58.131158 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d1785-0cad-4628-ba0e-32d3006d6de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:58.133339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:58.231926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d1785-0cad-4628-ba0e-32d3006d6de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:58.233064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 102/202] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:47:58.234671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d1785-0cad-4628-ba0e-32d3006d6de2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:58.235260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef46f3a-c14f-4cd7-9301-6122bdd4886a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:58.322987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:58.351609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 101/203] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:47:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:47:58.352724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d1785-0cad-4628-ba0e-32d3006d6de2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:58.372691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef46f3a-c14f-4cd7-9301-6122bdd4886a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:58.402070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:58.418542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944e60a-c5cf-4823-99eb-5d182b66a0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:58.419254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944e60a-c5cf-4823-99eb-5d182b66a0ad None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:58.419584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944e60a-c5cf-4823-99eb-5d182b66a0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:58.419896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944e60a-c5cf-4823-99eb-5d182b66a0ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:58.420689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944e60a-c5cf-4823-99eb-5d182b66a0ad None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=f456bff4d42f4ff194d2ac607af555db, user.id=31dfdfab58dc4039933640d2aa79310a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:58.421052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944e60a-c5cf-4823-99eb-5d182b66a0ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:58.423311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944e60a-c5cf-4823-99eb-5d182b66a0ad None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:58.429957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 103/204] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:58 2022] GET /identity/v3/role_assignments?user.id=31dfdfab58dc4039933640d2aa79310a&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:58.430594 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944e60a-c5cf-4823-99eb-5d182b66a0ad None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:58.505594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d1785-0cad-4628-ba0e-32d3006d6de2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:58.570400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5530b1a-24fd-45ed-b330-4fdacdaf344c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:58.571103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5530b1a-24fd-45ed-b330-4fdacdaf344c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:58.571411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5530b1a-24fd-45ed-b330-4fdacdaf344c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:58.571708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5530b1a-24fd-45ed-b330-4fdacdaf344c None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:58.576067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5530b1a-24fd-45ed-b330-4fdacdaf344c None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:58.579285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5530b1a-24fd-45ed-b330-4fdacdaf344c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:58.590328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 102/205] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:58 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:58.592629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944e60a-c5cf-4823-99eb-5d182b66a0ad None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:58.604071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5530b1a-24fd-45ed-b330-4fdacdaf344c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:58.662944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60bb5-d4d6-4c94-ba79-4dc6a15e26f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:58.664631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:58.665185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60bb5-d4d6-4c94-ba79-4dc6a15e26f2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:58.666504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60bb5-d4d6-4c94-ba79-4dc6a15e26f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:58.666982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60bb5-d4d6-4c94-ba79-4dc6a15e26f2 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:58.669288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec60bb5-d4d6-4c94-ba79-4dc6a15e26f2 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:58.671531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec60bb5-d4d6-4c94-ba79-4dc6a15e26f2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:58.673528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 104/206] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:47:58 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:58.679458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60bb5-d4d6-4c94-ba79-4dc6a15e26f2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:58.753596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5530b1a-24fd-45ed-b330-4fdacdaf344c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:58.778942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:58.818144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78420b15-4b29-4f2c-80d4-3164d0fe6600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:58.819026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78420b15-4b29-4f2c-80d4-3164d0fe6600 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:58.819299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78420b15-4b29-4f2c-80d4-3164d0fe6600 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:58.819565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78420b15-4b29-4f2c-80d4-3164d0fe6600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:58.820289 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78420b15-4b29-4f2c-80d4-3164d0fe6600 None admin] RBAC: Adding query filter params (role.id=31cc9f3a35594d40924277f4778c9f03, scope.project.id=f456bff4d42f4ff194d2ac607af555db, user.id=31dfdfab58dc4039933640d2aa79310a) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:58.820590 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78420b15-4b29-4f2c-80d4-3164d0fe6600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:58.822024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78420b15-4b29-4f2c-80d4-3164d0fe6600 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:58.834892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 103/207] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:58 2022] GET /identity/v3/role_assignments?user.id=31dfdfab58dc4039933640d2aa79310a&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=31cc9f3a35594d40924277f4778c9f03 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:47:58.875642 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60bb5-d4d6-4c94-ba79-4dc6a15e26f2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:58.926827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece1365-e4d7-46b0-9630-1fe541a0f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:58.927546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece1365-e4d7-46b0-9630-1fe541a0f2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:58.927828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece1365-e4d7-46b0-9630-1fe541a0f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:58.928101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece1365-e4d7-46b0-9630-1fe541a0f2e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:58.928824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece1365-e4d7-46b0-9630-1fe541a0f2e0 None admin] RBAC: Adding query filter params (role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086, user.id=fbfa2ddf17434fa09745d5a1fa32cb4f) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:47:58.929127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece1365-e4d7-46b0-9630-1fe541a0f2e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:58.930563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece1365-e4d7-46b0-9630-1fe541a0f2e0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:58.941367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 105/208] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:47:58 2022] GET /identity/v3/role_assignments?user.id=fbfa2ddf17434fa09745d5a1fa32cb4f&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:59.237072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c136b-3a39-4fe7-94d9-1ea40e78ebf3 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:59.237415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c136b-3a39-4fe7-94d9-1ea40e78ebf3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:59.237522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c136b-3a39-4fe7-94d9-1ea40e78ebf3 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:59.239036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 104/209] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:59.244576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996cae-721f-43d5-b7a9-ea5a3b103a81 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:59.244866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996cae-721f-43d5-b7a9-ea5a3b103a81 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:59.245142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996cae-721f-43d5-b7a9-ea5a3b103a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:59.267650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f0996cae-721f-43d5-b7a9-ea5a3b103a81 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:59.268839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f0996cae-721f-43d5-b7a9-ea5a3b103a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:59.329641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096430c-505a-4f37-88e6-f88bcfe5ea3f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:59.330086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096430c-505a-4f37-88e6-f88bcfe5ea3f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:59.330551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096430c-505a-4f37-88e6-f88bcfe5ea3f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:59.332238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 105/210] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:47:59.337538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01faa0-ce27-4fcf-9e64-09521603e5eb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:59.337859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01faa0-ce27-4fcf-9e64-09521603e5eb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:59.338087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01faa0-ce27-4fcf-9e64-09521603e5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:59.357950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8b01faa0-ce27-4fcf-9e64-09521603e5eb None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:47:59.359375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 106/211] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:47:59.359800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8b01faa0-ce27-4fcf-9e64-09521603e5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:47:59.435113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 106/212] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:47:59.656551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0996cae-721f-43d5-b7a9-ea5a3b103a81 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:59.716448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:59.750890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01faa0-ce27-4fcf-9e64-09521603e5eb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:59.772626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0996cae-721f-43d5-b7a9-ea5a3b103a81 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:59.813092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fb9bf-a934-494e-b9a8-fa8c69b7bf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:59.814272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fb9bf-a934-494e-b9a8-fa8c69b7bf27 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:59.815056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fb9bf-a934-494e-b9a8-fa8c69b7bf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:59.815710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fb9bf-a934-494e-b9a8-fa8c69b7bf27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:59.828495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fb9bf-a934-494e-b9a8-fa8c69b7bf27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b22fb9bf-a934-494e-b9a8-fa8c69b7bf27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:47:59.835310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:47:59.836179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:47:59.836968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:47:59.840255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 107/213] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:47:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:47:59.845980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:47:59.847925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fb9bf-a934-494e-b9a8-fa8c69b7bf27 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:47:59.935805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01faa0-ce27-4fcf-9e64-09521603e5eb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:47:59.974143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5efe1-73ca-4f13-8c9c-605f4259c572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:47:59.974794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5efe1-73ca-4f13-8c9c-605f4259c572 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:47:59.975107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5efe1-73ca-4f13-8c9c-605f4259c572 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:47:59.975385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5efe1-73ca-4f13-8c9c-605f4259c572 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:47:59.976651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5efe1-73ca-4f13-8c9c-605f4259c572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:47:59.978245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5efe1-73ca-4f13-8c9c-605f4259c572 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:47:59.984465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 107/214] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:47:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:47:59.985804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5efe1-73ca-4f13-8c9c-605f4259c572 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:00.072517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fb9bf-a934-494e-b9a8-fa8c69b7bf27 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:00.089743 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:00.107047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5efe1-73ca-4f13-8c9c-605f4259c572 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:00.136499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:00.150222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872174d-82a8-49d5-b7d3-d8b593a25875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:00.150802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593624c9-f9e6-4af8-b83c-2e608862fea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:00.151448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593624c9-f9e6-4af8-b83c-2e608862fea9 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:00.151685 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872174d-82a8-49d5-b7d3-d8b593a25875 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:00.151783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593624c9-f9e6-4af8-b83c-2e608862fea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:00.151989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593624c9-f9e6-4af8-b83c-2e608862fea9 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:00.152444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872174d-82a8-49d5-b7d3-d8b593a25875 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:00.152650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593624c9-f9e6-4af8-b83c-2e608862fea9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:00.153104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872174d-82a8-49d5-b7d3-d8b593a25875 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:00.154003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872174d-82a8-49d5-b7d3-d8b593a25875 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:00.154104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593624c9-f9e6-4af8-b83c-2e608862fea9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:00.154648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872174d-82a8-49d5-b7d3-d8b593a25875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:00.156388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872174d-82a8-49d5-b7d3-d8b593a25875 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:00.168128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 108/215] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:47:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:00.180236 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872174d-82a8-49d5-b7d3-d8b593a25875 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:00.184016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 108/216] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:47:59 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:48:00.313430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872174d-82a8-49d5-b7d3-d8b593a25875 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:00.352267 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30e36c-2a86-499a-b0bf-9cc917e7fe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:00.352972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30e36c-2a86-499a-b0bf-9cc917e7fe62 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:00.353269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30e36c-2a86-499a-b0bf-9cc917e7fe62 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:00.353620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30e36c-2a86-499a-b0bf-9cc917e7fe62 None admin] PATH_INFO: `/v3/users/3b29f427ad814d69a7cdc5c8329db736` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:00.369277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30e36c-2a86-499a-b0bf-9cc917e7fe62 None admin] RBAC: Authorizing `identity:get_user(user_id=3b29f427ad814d69a7cdc5c8329db736)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:00.370808 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30e36c-2a86-499a-b0bf-9cc917e7fe62 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:00.372840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 109/217] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:00 2022] GET /identity/v3/users/3b29f427ad814d69a7cdc5c8329db736 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:00.379473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593624c9-f9e6-4af8-b83c-2e608862fea9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:00.492879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593624c9-f9e6-4af8-b83c-2e608862fea9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:00.531659 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152dce9-7d17-4f4c-b71d-7512edbc70da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:00.532310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152dce9-7d17-4f4c-b71d-7512edbc70da None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:00.532623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152dce9-7d17-4f4c-b71d-7512edbc70da None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:00.533000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152dce9-7d17-4f4c-b71d-7512edbc70da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:00.566244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:00.612629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152dce9-7d17-4f4c-b71d-7512edbc70da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:00.614115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152dce9-7d17-4f4c-b71d-7512edbc70da None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:00.650966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adf39e-8ed3-4154-8bc6-73231ce69f38 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:00.651269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adf39e-8ed3-4154-8bc6-73231ce69f38 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:00.655026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adf39e-8ed3-4154-8bc6-73231ce69f38 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:00.655026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 110/218] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:00.664437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60812207-83e6-419b-9eca-033adc578676 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:00.665035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60812207-83e6-419b-9eca-033adc578676 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:00.665035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60812207-83e6-419b-9eca-033adc578676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:00.697260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-60812207-83e6-419b-9eca-033adc578676 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:00.698461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-60812207-83e6-419b-9eca-033adc578676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:00.725953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 109/219] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:00.731869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152dce9-7d17-4f4c-b71d-7512edbc70da None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:00.751463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:00.812759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:00.855586 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152dce9-7d17-4f4c-b71d-7512edbc70da None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:00.872347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 111/220] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:00.893577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56be96-8030-4346-ba67-328a625caec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:00.893577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56be96-8030-4346-ba67-328a625caec3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:00.893577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56be96-8030-4346-ba67-328a625caec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:00.893577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56be96-8030-4346-ba67-328a625caec3 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:00.894827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56be96-8030-4346-ba67-328a625caec3 None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:00.896435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56be96-8030-4346-ba67-328a625caec3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:00.898348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 110/221] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:00 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:00.904087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60812207-83e6-419b-9eca-033adc578676 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:01.016846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60812207-83e6-419b-9eca-033adc578676 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:01.062085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9baaf-ee43-413b-9dc4-4e4d08c73e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:01.062705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9baaf-ee43-413b-9dc4-4e4d08c73e1b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:01.062986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9baaf-ee43-413b-9dc4-4e4d08c73e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:01.063285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9baaf-ee43-413b-9dc4-4e4d08c73e1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:01.064025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9baaf-ee43-413b-9dc4-4e4d08c73e1b None admin] RBAC: Adding query filter params (role.id=a0db5bd603fc4096917bbcd8c6932bc0, scope.project.id=f456bff4d42f4ff194d2ac607af555db, user.id=3b29f427ad814d69a7cdc5c8329db736) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:01.064336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9baaf-ee43-413b-9dc4-4e4d08c73e1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:01.065782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9baaf-ee43-413b-9dc4-4e4d08c73e1b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:01.071946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 112/222] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:00 2022] GET /identity/v3/role_assignments?user.id=3b29f427ad814d69a7cdc5c8329db736&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:01.084867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6731c8-0ba9-4773-ba29-2f73484eda72 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:01.085168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6731c8-0ba9-4773-ba29-2f73484eda72 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:01.085412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6731c8-0ba9-4773-ba29-2f73484eda72 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:01.086752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 111/223] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:01.092603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6934d-6e21-45f1-ae76-5516b4ba7321 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:01.093447 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6934d-6e21-45f1-ae76-5516b4ba7321 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:01.093709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6934d-6e21-45f1-ae76-5516b4ba7321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:01.120144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a3b6934d-6e21-45f1-ae76-5516b4ba7321 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:01.121301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a3b6934d-6e21-45f1-ae76-5516b4ba7321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:01.142045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:01.198096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 113/224] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:01.234395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6731c8-0ba9-4773-ba29-2f73484eda72 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:01.391843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6731c8-0ba9-4773-ba29-2f73484eda72 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:01.430551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:01.439178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d038a8-5833-4d1e-8c04-0af9c26a7eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:01.440131 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d038a8-5833-4d1e-8c04-0af9c26a7eeb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:01.440606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d038a8-5833-4d1e-8c04-0af9c26a7eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:01.441068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d038a8-5833-4d1e-8c04-0af9c26a7eeb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:01.456312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d038a8-5833-4d1e-8c04-0af9c26a7eeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-17d038a8-5833-4d1e-8c04-0af9c26a7eeb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:01.461612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:01.462465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:01.463285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:01.465613 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 112/225] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:48:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:01.472091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6934d-6e21-45f1-ae76-5516b4ba7321 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:01.506743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d038a8-5833-4d1e-8c04-0af9c26a7eeb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:01.597959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6934d-6e21-45f1-ae76-5516b4ba7321 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:01.639576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863c89b-e15c-4f8a-87b3-ff8de9e410a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:01.640196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c89b-e15c-4f8a-87b3-ff8de9e410a8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:01.640479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c89b-e15c-4f8a-87b3-ff8de9e410a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:01.640755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c89b-e15c-4f8a-87b3-ff8de9e410a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:01.641496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863c89b-e15c-4f8a-87b3-ff8de9e410a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:01.641799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863c89b-e15c-4f8a-87b3-ff8de9e410a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:01.643325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863c89b-e15c-4f8a-87b3-ff8de9e410a8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:01.652301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 114/226] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:01.658216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863c89b-e15c-4f8a-87b3-ff8de9e410a8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:01.674709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:01.701813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d038a8-5833-4d1e-8c04-0af9c26a7eeb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:01.765628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8d65c-daca-40d6-bc20-854d499e370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:01.767380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8d65c-daca-40d6-bc20-854d499e370b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:01.768472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8d65c-daca-40d6-bc20-854d499e370b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:01.770160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8d65c-daca-40d6-bc20-854d499e370b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:01.788673 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8d65c-daca-40d6-bc20-854d499e370b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3ac8d65c-daca-40d6-bc20-854d499e370b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:01.796845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:01.797784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:01.798578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:01.802886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 113/227] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:48:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:01.810520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8d65c-daca-40d6-bc20-854d499e370b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:01.856907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863c89b-e15c-4f8a-87b3-ff8de9e410a8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:01.912335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:01.928025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11da17e-febf-4d8c-84d5-8c3359745a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:01.930083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11da17e-febf-4d8c-84d5-8c3359745a29 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:01.931536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11da17e-febf-4d8c-84d5-8c3359745a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:01.932014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11da17e-febf-4d8c-84d5-8c3359745a29 None admin] PATH_INFO: `/v3/users/31dfdfab58dc4039933640d2aa79310a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:01.947014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11da17e-febf-4d8c-84d5-8c3359745a29 None admin] RBAC: Authorizing `identity:get_user(user_id=31dfdfab58dc4039933640d2aa79310a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:01.955402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11da17e-febf-4d8c-84d5-8c3359745a29 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:01.957308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 115/228] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:01 2022] GET /identity/v3/users/31dfdfab58dc4039933640d2aa79310a => generated 283 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:01.964190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11da17e-febf-4d8c-84d5-8c3359745a29 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:01.991736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8d65c-daca-40d6-bc20-854d499e370b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:02.029384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee0c25-19e1-4e31-9745-b0667bbd2677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:02.031213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee0c25-19e1-4e31-9745-b0667bbd2677 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:02.031698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee0c25-19e1-4e31-9745-b0667bbd2677 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:02.032219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee0c25-19e1-4e31-9745-b0667bbd2677 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:02.033177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee0c25-19e1-4e31-9745-b0667bbd2677 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:02.033726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee0c25-19e1-4e31-9745-b0667bbd2677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:02.035435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee0c25-19e1-4e31-9745-b0667bbd2677 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:02.044180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 114/229] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:02.050606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee0c25-19e1-4e31-9745-b0667bbd2677 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:02.153706 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:02.158013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee0c25-19e1-4e31-9745-b0667bbd2677 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:02.176954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:02.195412 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11da17e-febf-4d8c-84d5-8c3359745a29 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:02.211023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4ce87-41f9-4f9f-a57d-424be530453f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:02.213680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4ce87-41f9-4f9f-a57d-424be530453f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:02.214162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4ce87-41f9-4f9f-a57d-424be530453f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:02.214631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4ce87-41f9-4f9f-a57d-424be530453f None admin] PATH_INFO: `/v3/users/31dfdfab58dc4039933640d2aa79310a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:02.217286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4ce87-41f9-4f9f-a57d-424be530453f None admin] RBAC: Authorizing `identity:get_user(user_id=31dfdfab58dc4039933640d2aa79310a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:02.220486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4ce87-41f9-4f9f-a57d-424be530453f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:02.222502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 115/230] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:02 2022] GET /identity/v3/users/31dfdfab58dc4039933640d2aa79310a => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:02.228563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4ce87-41f9-4f9f-a57d-424be530453f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:02.263280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d063b9a-ec7f-4b37-b2e9-61279152319c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:02.264667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d063b9a-ec7f-4b37-b2e9-61279152319c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:02.265176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d063b9a-ec7f-4b37-b2e9-61279152319c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:02.265731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d063b9a-ec7f-4b37-b2e9-61279152319c None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:02.267660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d063b9a-ec7f-4b37-b2e9-61279152319c None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:02.269521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d063b9a-ec7f-4b37-b2e9-61279152319c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:02.271462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 116/231] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:01 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:02.277770 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d063b9a-ec7f-4b37-b2e9-61279152319c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:02.369670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4ce87-41f9-4f9f-a57d-424be530453f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:02.427668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d063b9a-ec7f-4b37-b2e9-61279152319c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:02.446572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53380e59-8269-4971-a667-5eca13083747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:02.447604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380e59-8269-4971-a667-5eca13083747 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:02.448182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380e59-8269-4971-a667-5eca13083747 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:02.448790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380e59-8269-4971-a667-5eca13083747 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:02.451301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53380e59-8269-4971-a667-5eca13083747 None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:02.453690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53380e59-8269-4971-a667-5eca13083747 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:02.456287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 116/232] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:02 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:02.463282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53380e59-8269-4971-a667-5eca13083747 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:02.512634 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212d552-cbac-441e-8792-62590b9a14a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:02.513315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d552-cbac-441e-8792-62590b9a14a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:02.513594 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d552-cbac-441e-8792-62590b9a14a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:02.513867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d552-cbac-441e-8792-62590b9a14a5 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db/users/31dfdfab58dc4039933640d2aa79310a/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:02.516362 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212d552-cbac-441e-8792-62590b9a14a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f456bff4d42f4ff194d2ac607af555db, user_id=31dfdfab58dc4039933640d2aa79310a, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:02.517904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212d552-cbac-441e-8792-62590b9a14a5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:02.527379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:02.558943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 117/233] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:48:02 2022] PUT /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db/users/31dfdfab58dc4039933640d2aa79310a/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:48:02.600260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53380e59-8269-4971-a667-5eca13083747 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:02.654866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d5a79-47c8-4d79-a8b3-6a70caa31789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:02.656703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d5a79-47c8-4d79-a8b3-6a70caa31789 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:02.657176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d5a79-47c8-4d79-a8b3-6a70caa31789 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:02.657661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d5a79-47c8-4d79-a8b3-6a70caa31789 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db/users/31dfdfab58dc4039933640d2aa79310a/roles/31cc9f3a35594d40924277f4778c9f03` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:02.660147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d5a79-47c8-4d79-a8b3-6a70caa31789 None admin] RBAC: Authorizing `identity:create_grant(project_id=f456bff4d42f4ff194d2ac607af555db, user_id=31dfdfab58dc4039933640d2aa79310a, role_id=31cc9f3a35594d40924277f4778c9f03)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:02.661822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d5a79-47c8-4d79-a8b3-6a70caa31789 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:02.685369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 117/234] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:48:02 2022] PUT /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db/users/31dfdfab58dc4039933640d2aa79310a/roles/31cc9f3a35594d40924277f4778c9f03 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:48:02.792955 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d12707-acd6-4981-836c-ee6eeb23feb3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:02.793464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d12707-acd6-4981-836c-ee6eeb23feb3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:02.793568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d12707-acd6-4981-836c-ee6eeb23feb3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:02.794954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 118/235] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:02.802642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eba23-b16d-4b33-a852-65077df39218 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:02.803309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eba23-b16d-4b33-a852-65077df39218 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:02.803780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eba23-b16d-4b33-a852-65077df39218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:02.832412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-310eba23-b16d-4b33-a852-65077df39218 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:02.833993 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-310eba23-b16d-4b33-a852-65077df39218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:02.913059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20522d3-d504-48e3-9c8b-0818a439efc2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:02.913634 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20522d3-d504-48e3-9c8b-0818a439efc2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:02.913912 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20522d3-d504-48e3-9c8b-0818a439efc2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:02.915536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 119/236] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:02.922382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 118/237] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 29 18:48:02.923287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7d45b-616b-475c-a1bf-abc0d1b2e19c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:02.923638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7d45b-616b-475c-a1bf-abc0d1b2e19c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:02.924014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7d45b-616b-475c-a1bf-abc0d1b2e19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:02.958238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bed7d45b-616b-475c-a1bf-abc0d1b2e19c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:02.959978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bed7d45b-616b-475c-a1bf-abc0d1b2e19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:02.962321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:03.056100 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 120/238] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:03.400590 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eba23-b16d-4b33-a852-65077df39218 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:03.419111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185f0c4-d018-4755-9d45-aabb5f603fdd None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:03.419397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185f0c4-d018-4755-9d45-aabb5f603fdd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:03.419672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185f0c4-d018-4755-9d45-aabb5f603fdd None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:03.421557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 121/239] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:48:03.431342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0177a2-571c-466a-8960-25f92183a309 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:03.431627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0177a2-571c-466a-8960-25f92183a309 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:03.431901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0177a2-571c-466a-8960-25f92183a309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:03.452649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:03.460060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bd0177a2-571c-466a-8960-25f92183a309 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:03.462680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bd0177a2-571c-466a-8960-25f92183a309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:03.520299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eba23-b16d-4b33-a852-65077df39218 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:03.559649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4697ddb-2629-4c6e-b3f1-2f6180906d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:03.560397 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4697ddb-2629-4c6e-b3f1-2f6180906d5b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:03.560724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4697ddb-2629-4c6e-b3f1-2f6180906d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:03.561056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4697ddb-2629-4c6e-b3f1-2f6180906d5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:03.562085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 122/240] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:03.562304 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4697ddb-2629-4c6e-b3f1-2f6180906d5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:03.563488 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0177a2-571c-466a-8960-25f92183a309 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:03.568885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4697ddb-2629-4c6e-b3f1-2f6180906d5b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:03.570704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 119/241] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:03.575929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4697ddb-2629-4c6e-b3f1-2f6180906d5b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:03.639439 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:03.680388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0177a2-571c-466a-8960-25f92183a309 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:03.729898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99f72d-93d4-42a4-853e-398280e71c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:03.730536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99f72d-93d4-42a4-853e-398280e71c39 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:03.730813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99f72d-93d4-42a4-853e-398280e71c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:03.731108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99f72d-93d4-42a4-853e-398280e71c39 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:03.752442 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99f72d-93d4-42a4-853e-398280e71c39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4c99f72d-93d4-42a4-853e-398280e71c39 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:03.757662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:03.758552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:03.759369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:03.760473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 123/242] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:03.771574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99f72d-93d4-42a4-853e-398280e71c39 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:03.794570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4697ddb-2629-4c6e-b3f1-2f6180906d5b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:03.804067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:03.859952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9c14b-1a3e-407f-95fa-e677f7ef8c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:03.860569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9c14b-1a3e-407f-95fa-e677f7ef8c00 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:03.860845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9c14b-1a3e-407f-95fa-e677f7ef8c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:03.861228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9c14b-1a3e-407f-95fa-e677f7ef8c00 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:03.861886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9c14b-1a3e-407f-95fa-e677f7ef8c00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:03.864744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9c14b-1a3e-407f-95fa-e677f7ef8c00 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:03.907731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 120/243] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:48:03 2022] POST /identity/v3/users => generated 293 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:48:03.940692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9c14b-1a3e-407f-95fa-e677f7ef8c00 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:03.968436 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99f72d-93d4-42a4-853e-398280e71c39 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:04.003697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:04.013433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e5879-17a8-43e1-b56b-e518097796ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:04.014056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e5879-17a8-43e1-b56b-e518097796ae None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:04.014331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e5879-17a8-43e1-b56b-e518097796ae None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:04.014645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e5879-17a8-43e1-b56b-e518097796ae None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:04.015346 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e5879-17a8-43e1-b56b-e518097796ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:04.015697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e5879-17a8-43e1-b56b-e518097796ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:04.017211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e5879-17a8-43e1-b56b-e518097796ae None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:04.027122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 124/244] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:04.034328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e5879-17a8-43e1-b56b-e518097796ae None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:04.153826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9c14b-1a3e-407f-95fa-e677f7ef8c00 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:04.162620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e5879-17a8-43e1-b56b-e518097796ae None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:04.200840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30663ca-1b6b-4b6b-aa96-bd73f124e09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:04.201488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30663ca-1b6b-4b6b-aa96-bd73f124e09e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:04.201608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52126d12-0352-41e5-b509-481aeb0a7934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:04.201783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30663ca-1b6b-4b6b-aa96-bd73f124e09e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:04.202063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30663ca-1b6b-4b6b-aa96-bd73f124e09e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:04.202235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52126d12-0352-41e5-b509-481aeb0a7934 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:04.202484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52126d12-0352-41e5-b509-481aeb0a7934 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:04.202750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52126d12-0352-41e5-b509-481aeb0a7934 None admin] PATH_INFO: `/v3/users/cc93504afac5417ab0e7e60a1c5be6c5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:04.218914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30663ca-1b6b-4b6b-aa96-bd73f124e09e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:04.220246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52126d12-0352-41e5-b509-481aeb0a7934 None admin] RBAC: Authorizing `identity:get_user(user_id=cc93504afac5417ab0e7e60a1c5be6c5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:04.221790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52126d12-0352-41e5-b509-481aeb0a7934 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b30663ca-1b6b-4b6b-aa96-bd73f124e09e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:04.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:04.227083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:04.229000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:04.230517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 125/245] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:04 2022] GET /identity/v3/users/cc93504afac5417ab0e7e60a1c5be6c5 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:04.230648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 121/246] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:04.230778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52126d12-0352-41e5-b509-481aeb0a7934 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:04.238101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30663ca-1b6b-4b6b-aa96-bd73f124e09e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:04.265050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:04.302164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:04.369932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30663ca-1b6b-4b6b-aa96-bd73f124e09e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:04.370291 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52126d12-0352-41e5-b509-481aeb0a7934 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:04.412544 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b2ecf-4c0c-4883-a70d-d9fb1b97e4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:04.413159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b2ecf-4c0c-4883-a70d-d9fb1b97e4ee None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:04.413437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b2ecf-4c0c-4883-a70d-d9fb1b97e4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:04.413799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b2ecf-4c0c-4883-a70d-d9fb1b97e4ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:04.416627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea072ca1-201e-47e7-b873-d8c1c5f085bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:04.417411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea072ca1-201e-47e7-b873-d8c1c5f085bb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:04.418769 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea072ca1-201e-47e7-b873-d8c1c5f085bb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:04.419234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea072ca1-201e-47e7-b873-d8c1c5f085bb None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:04.420050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea072ca1-201e-47e7-b873-d8c1c5f085bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:04.420493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea072ca1-201e-47e7-b873-d8c1c5f085bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:04.422580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea072ca1-201e-47e7-b873-d8c1c5f085bb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:04.435705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 122/247] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:04.441976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea072ca1-201e-47e7-b873-d8c1c5f085bb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:04.499086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b2ecf-4c0c-4883-a70d-d9fb1b97e4ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:04.500497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b2ecf-4c0c-4883-a70d-d9fb1b97e4ee None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:04.537669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:04.556244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea072ca1-201e-47e7-b873-d8c1c5f085bb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:04.557955 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:04.597244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8a75f-d51f-462e-a0fb-67998383d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:04.598123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a75f-d51f-462e-a0fb-67998383d7ea None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:04.598541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a75f-d51f-462e-a0fb-67998383d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:04.598953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a75f-d51f-462e-a0fb-67998383d7ea None admin] PATH_INFO: `/v3/users/3b29f427ad814d69a7cdc5c8329db736` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:04.600257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8a75f-d51f-462e-a0fb-67998383d7ea None admin] RBAC: Authorizing `identity:get_user(user_id=3b29f427ad814d69a7cdc5c8329db736)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:04.602041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8a75f-d51f-462e-a0fb-67998383d7ea None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:04.604172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 123/248] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:04 2022] GET /identity/v3/users/3b29f427ad814d69a7cdc5c8329db736 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:04.612242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8a75f-d51f-462e-a0fb-67998383d7ea None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:04.622405 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 126/249] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:04.627949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b2ecf-4c0c-4883-a70d-d9fb1b97e4ee None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:04.746211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8a75f-d51f-462e-a0fb-67998383d7ea None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:04.756532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b2ecf-4c0c-4883-a70d-d9fb1b97e4ee None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:04.786010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d235881-fb84-44b6-8c1a-d392e9c7f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:04.786647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d235881-fb84-44b6-8c1a-d392e9c7f7ca None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:04.786924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d235881-fb84-44b6-8c1a-d392e9c7f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:04.787241 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d235881-fb84-44b6-8c1a-d392e9c7f7ca None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:04.788383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d235881-fb84-44b6-8c1a-d392e9c7f7ca None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:04.789819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d235881-fb84-44b6-8c1a-d392e9c7f7ca None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:04.791895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 124/250] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:04 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:04.797263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d235881-fb84-44b6-8c1a-d392e9c7f7ca None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:04.811233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53808018-efd2-416b-a5ef-16437b89b073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:04.811913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53808018-efd2-416b-a5ef-16437b89b073 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:04.812250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53808018-efd2-416b-a5ef-16437b89b073 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:04.812530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53808018-efd2-416b-a5ef-16437b89b073 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:04.813689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53808018-efd2-416b-a5ef-16437b89b073 None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:04.815579 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53808018-efd2-416b-a5ef-16437b89b073 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:04.816967 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 127/251] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:04 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:04.829319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53808018-efd2-416b-a5ef-16437b89b073 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:04.847989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:04.862807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:04.961042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d235881-fb84-44b6-8c1a-d392e9c7f7ca None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:04.982680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53808018-efd2-416b-a5ef-16437b89b073 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:05.028652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e4806-27c6-4990-93ff-a6ab1692ae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:05.029438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e4806-27c6-4990-93ff-a6ab1692ae95 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:05.030022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e4806-27c6-4990-93ff-a6ab1692ae95 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:05.030526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e4806-27c6-4990-93ff-a6ab1692ae95 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db/users/3b29f427ad814d69a7cdc5c8329db736/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:05.034047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e4806-27c6-4990-93ff-a6ab1692ae95 None admin] RBAC: Authorizing `identity:create_grant(project_id=f456bff4d42f4ff194d2ac607af555db, user_id=3b29f427ad814d69a7cdc5c8329db736, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:05.035953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e4806-27c6-4990-93ff-a6ab1692ae95 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:05.037663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d842f8-c653-4f81-9402-9a78c4665b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:05.038274 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d842f8-c653-4f81-9402-9a78c4665b8a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:05.038676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d842f8-c653-4f81-9402-9a78c4665b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:05.038950 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d842f8-c653-4f81-9402-9a78c4665b8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:05.039957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d842f8-c653-4f81-9402-9a78c4665b8a None admin] RBAC: Adding query filter params (role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d, scope.project.id=f456bff4d42f4ff194d2ac607af555db, user.id=cc93504afac5417ab0e7e60a1c5be6c5) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:05.040475 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d842f8-c653-4f81-9402-9a78c4665b8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:05.041850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d842f8-c653-4f81-9402-9a78c4665b8a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:05.051359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 128/252] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:04 2022] GET /identity/v3/role_assignments?user.id=cc93504afac5417ab0e7e60a1c5be6c5&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:05.051359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a6d4e-73ff-40c0-9747-9dcea68bee42 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:05.051359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a6d4e-73ff-40c0-9747-9dcea68bee42 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:05.051359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a6d4e-73ff-40c0-9747-9dcea68bee42 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:05.051359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 129/253] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:05.056366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcc9e1-6fed-4028-9e8a-bf34a1fdade2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:05.056751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcc9e1-6fed-4028-9e8a-bf34a1fdade2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:05.057041 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcc9e1-6fed-4028-9e8a-bf34a1fdade2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:05.058625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 130/254] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:05.058918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 125/255] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:48:04 2022] PUT /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db/users/3b29f427ad814d69a7cdc5c8329db736/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:48:05.060027 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f6480-90d7-4903-be97-82580faf00b2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:05.060027 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f6480-90d7-4903-be97-82580faf00b2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:05.060295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f6480-90d7-4903-be97-82580faf00b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:05.085311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ff279-3318-452a-aba8-b3c8511c70b6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:05.085917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ff279-3318-452a-aba8-b3c8511c70b6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:05.086405 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ff279-3318-452a-aba8-b3c8511c70b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:05.092692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-dd4f6480-90d7-4903-be97-82580faf00b2 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:05.094145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dd4f6480-90d7-4903-be97-82580faf00b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:05.130315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1c0ff279-3318-452a-aba8-b3c8511c70b6 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:05.131544 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1c0ff279-3318-452a-aba8-b3c8511c70b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:05.261495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 131/256] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:05.291486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 126/257] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:05.568840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f6480-90d7-4903-be97-82580faf00b2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:05.581367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:05.700970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f6480-90d7-4903-be97-82580faf00b2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:05.744345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747968d9-9ce1-457a-afaa-dbcb025c1923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:05.744986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747968d9-9ce1-457a-afaa-dbcb025c1923 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:05.745267 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747968d9-9ce1-457a-afaa-dbcb025c1923 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:05.745539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747968d9-9ce1-457a-afaa-dbcb025c1923 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:05.764186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747968d9-9ce1-457a-afaa-dbcb025c1923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-747968d9-9ce1-457a-afaa-dbcb025c1923 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:05.769051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:05.769941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:05.771402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 132/258] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:48:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:05.776135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ff279-3318-452a-aba8-b3c8511c70b6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:05.789213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:05.808642 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747968d9-9ce1-457a-afaa-dbcb025c1923 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:05.931561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:05.965394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ff279-3318-452a-aba8-b3c8511c70b6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:06.000972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747968d9-9ce1-457a-afaa-dbcb025c1923 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:06.015683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518880c3-9104-4e98-9113-9e5fb0c3313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:06.017144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518880c3-9104-4e98-9113-9e5fb0c3313b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:06.017899 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518880c3-9104-4e98-9113-9e5fb0c3313b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:06.018366 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518880c3-9104-4e98-9113-9e5fb0c3313b None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:06.019586 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518880c3-9104-4e98-9113-9e5fb0c3313b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:06.020200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518880c3-9104-4e98-9113-9e5fb0c3313b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:06.021976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518880c3-9104-4e98-9113-9e5fb0c3313b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:06.033554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 127/259] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:06.039853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518880c3-9104-4e98-9113-9e5fb0c3313b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:06.127734 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ab58e-b327-464b-8bc3-23ff3b177258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:06.128445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ab58e-b327-464b-8bc3-23ff3b177258 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:06.128668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ab58e-b327-464b-8bc3-23ff3b177258 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:06.128941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ab58e-b327-464b-8bc3-23ff3b177258 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:06.172120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ab58e-b327-464b-8bc3-23ff3b177258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:06.174514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-891ab58e-b327-464b-8bc3-23ff3b177258 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:06.177162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:06.178068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:06.178848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 133/260] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:48:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:06.186060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ab58e-b327-464b-8bc3-23ff3b177258 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:06.229927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518880c3-9104-4e98-9113-9e5fb0c3313b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:06.304435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9464663-b293-47c0-9ced-5021dc527357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:06.305113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9464663-b293-47c0-9ced-5021dc527357 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:06.305456 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9464663-b293-47c0-9ced-5021dc527357 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:06.305737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9464663-b293-47c0-9ced-5021dc527357 None admin] PATH_INFO: `/v3/users/31dfdfab58dc4039933640d2aa79310a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:06.307261 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9464663-b293-47c0-9ced-5021dc527357 None admin] RBAC: Authorizing `identity:get_user(user_id=31dfdfab58dc4039933640d2aa79310a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:06.308799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9464663-b293-47c0-9ced-5021dc527357 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:06.310413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 128/261] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:06 2022] GET /identity/v3/users/31dfdfab58dc4039933640d2aa79310a => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:06.321890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9464663-b293-47c0-9ced-5021dc527357 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:06.415174 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ab58e-b327-464b-8bc3-23ff3b177258 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:06.477676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:06.487440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3a480-98db-40f9-a886-26e3a6026e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:06.488349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3a480-98db-40f9-a886-26e3a6026e75 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:06.488853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3a480-98db-40f9-a886-26e3a6026e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:06.489321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3a480-98db-40f9-a886-26e3a6026e75 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:06.490205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3a480-98db-40f9-a886-26e3a6026e75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:06.491247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3a480-98db-40f9-a886-26e3a6026e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:06.492840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3a480-98db-40f9-a886-26e3a6026e75 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:06.514193 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 134/262] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:06.525569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3a480-98db-40f9-a886-26e3a6026e75 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:06.559057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9464663-b293-47c0-9ced-5021dc527357 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:06.659718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8cb55-2b42-446a-85f0-1478b444f5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:06.660351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8cb55-2b42-446a-85f0-1478b444f5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:06.660634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8cb55-2b42-446a-85f0-1478b444f5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:06.660965 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8cb55-2b42-446a-85f0-1478b444f5f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:06.683618 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:06.709823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3a480-98db-40f9-a886-26e3a6026e75 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:06.772575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdd472-f88f-4173-bb20-3bee6c743c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:06.773235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdd472-f88f-4173-bb20-3bee6c743c01 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:06.773511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdd472-f88f-4173-bb20-3bee6c743c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:06.773788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdd472-f88f-4173-bb20-3bee6c743c01 None admin] PATH_INFO: `/v3/users/31dfdfab58dc4039933640d2aa79310a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:06.776036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdd472-f88f-4173-bb20-3bee6c743c01 None admin] RBAC: Authorizing `identity:get_user(user_id=31dfdfab58dc4039933640d2aa79310a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:06.777493 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdd472-f88f-4173-bb20-3bee6c743c01 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:06.780894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 135/263] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:06 2022] GET /identity/v3/users/31dfdfab58dc4039933640d2aa79310a => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:06.786941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdd472-f88f-4173-bb20-3bee6c743c01 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:06.810904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8cb55-2b42-446a-85f0-1478b444f5f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:06.812968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8cb55-2b42-446a-85f0-1478b444f5f3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:06.912332 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:06.961711 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 129/264] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:06.967319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8cb55-2b42-446a-85f0-1478b444f5f3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:06.987987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdd472-f88f-4173-bb20-3bee6c743c01 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:07.044777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1717ec8-89a9-4b8d-b550-b88888f909ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:07.046270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1717ec8-89a9-4b8d-b550-b88888f909ab None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:07.046741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1717ec8-89a9-4b8d-b550-b88888f909ab None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:07.047246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1717ec8-89a9-4b8d-b550-b88888f909ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:07.055304 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:07.165886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8cb55-2b42-446a-85f0-1478b444f5f3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:07.210866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830f7bb-50c3-4ef9-8537-637f7cfd8f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:07.211833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830f7bb-50c3-4ef9-8537-637f7cfd8f0c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:07.213311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830f7bb-50c3-4ef9-8537-637f7cfd8f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:07.213311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830f7bb-50c3-4ef9-8537-637f7cfd8f0c None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:07.214485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830f7bb-50c3-4ef9-8537-637f7cfd8f0c None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:07.216815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830f7bb-50c3-4ef9-8537-637f7cfd8f0c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:07.220157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 130/265] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:06 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:07.228122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1717ec8-89a9-4b8d-b550-b88888f909ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:07.229900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1717ec8-89a9-4b8d-b550-b88888f909ab None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:07.236547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830f7bb-50c3-4ef9-8537-637f7cfd8f0c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:07.348781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830f7bb-50c3-4ef9-8537-637f7cfd8f0c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:07.363025 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:07.380031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:07.390651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b853c5c-d897-4cf3-83e2-c5a80f3e9aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:07.391549 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b853c5c-d897-4cf3-83e2-c5a80f3e9aad None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:07.392245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b853c5c-d897-4cf3-83e2-c5a80f3e9aad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:07.392705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b853c5c-d897-4cf3-83e2-c5a80f3e9aad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:07.393602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b853c5c-d897-4cf3-83e2-c5a80f3e9aad None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=f456bff4d42f4ff194d2ac607af555db, user.id=31dfdfab58dc4039933640d2aa79310a) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:07.394066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b853c5c-d897-4cf3-83e2-c5a80f3e9aad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:07.395747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b853c5c-d897-4cf3-83e2-c5a80f3e9aad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:07.402308 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 131/266] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:07 2022] GET /identity/v3/role_assignments?user.id=31dfdfab58dc4039933640d2aa79310a&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:07.456075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 136/267] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:07.462204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b853c5c-d897-4cf3-83e2-c5a80f3e9aad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:07.592331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7dec7-d4a2-4ccf-9574-158227909168 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:07.592817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7dec7-d4a2-4ccf-9574-158227909168 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:07.593435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7dec7-d4a2-4ccf-9574-158227909168 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:07.599408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 137/268] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:07.604450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27310d-8eba-4b8a-b5df-0e9379147ece None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:07.604811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27310d-8eba-4b8a-b5df-0e9379147ece None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:07.605160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27310d-8eba-4b8a-b5df-0e9379147ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:07.609834 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b853c5c-d897-4cf3-83e2-c5a80f3e9aad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:07.641151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5b27310d-8eba-4b8a-b5df-0e9379147ece None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:07.642470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5b27310d-8eba-4b8a-b5df-0e9379147ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:07.657299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bedaad-f7e4-49e0-8fc9-1ba3c99e9972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:07.658368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bedaad-f7e4-49e0-8fc9-1ba3c99e9972 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:07.658815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bedaad-f7e4-49e0-8fc9-1ba3c99e9972 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:07.659274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bedaad-f7e4-49e0-8fc9-1ba3c99e9972 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:07.661087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bedaad-f7e4-49e0-8fc9-1ba3c99e9972 None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:07.662734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bedaad-f7e4-49e0-8fc9-1ba3c99e9972 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:07.664863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 132/269] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:07 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:07.671712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bedaad-f7e4-49e0-8fc9-1ba3c99e9972 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:07.738581 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:07.775515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 138/270] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:07.777242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b85f-5d6a-4a8a-a0a5-3f53a945b3c9 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:07.777516 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b85f-5d6a-4a8a-a0a5-3f53a945b3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:07.777862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b85f-5d6a-4a8a-a0a5-3f53a945b3c9 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:07.779292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 139/271] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:07.785039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5204-16fa-4fb2-a3ce-6eb8db6d5afa None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:07.785369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5204-16fa-4fb2-a3ce-6eb8db6d5afa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:07.785713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5204-16fa-4fb2-a3ce-6eb8db6d5afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:07.807947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-02eb5204-16fa-4fb2-a3ce-6eb8db6d5afa None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:07.809562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-02eb5204-16fa-4fb2-a3ce-6eb8db6d5afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:07.810174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bedaad-f7e4-49e0-8fc9-1ba3c99e9972 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:07.860017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e229a4-3558-4c42-babf-d311d23e7656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:07.861063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e229a4-3558-4c42-babf-d311d23e7656 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:07.861556 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e229a4-3558-4c42-babf-d311d23e7656 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:07.862067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e229a4-3558-4c42-babf-d311d23e7656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:07.863063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e229a4-3558-4c42-babf-d311d23e7656 None admin] RBAC: Adding query filter params (role.id=31cc9f3a35594d40924277f4778c9f03, scope.project.id=f456bff4d42f4ff194d2ac607af555db, user.id=31dfdfab58dc4039933640d2aa79310a) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:07.863584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e229a4-3558-4c42-babf-d311d23e7656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:07.867492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e229a4-3558-4c42-babf-d311d23e7656 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:07.875432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 133/272] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:07 2022] GET /identity/v3/role_assignments?user.id=31dfdfab58dc4039933640d2aa79310a&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=31cc9f3a35594d40924277f4778c9f03 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:07.877178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4ab23-799d-4da0-a643-902a34a51a58 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:07.877655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4ab23-799d-4da0-a643-902a34a51a58 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:07.878115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4ab23-799d-4da0-a643-902a34a51a58 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:07.879558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 134/273] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:07.891031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c745ce-35c1-4ca4-b1c0-8f67fb34009e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:07.891031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c745ce-35c1-4ca4-b1c0-8f67fb34009e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:07.891031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c745ce-35c1-4ca4-b1c0-8f67fb34009e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:07.921847 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-01c745ce-35c1-4ca4-b1c0-8f67fb34009e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:07.923215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-01c745ce-35c1-4ca4-b1c0-8f67fb34009e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:07.937788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 140/274] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:08.025115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 135/275] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:08.043630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c071a2-2711-4b36-9953-83dcfce6258c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:08.043930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c071a2-2711-4b36-9953-83dcfce6258c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:08.044184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c071a2-2711-4b36-9953-83dcfce6258c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:08.045530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 141/276] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:08.054316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d5f1c-dc9f-440a-befd-302b449aaa61 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:08.054641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d5f1c-dc9f-440a-befd-302b449aaa61 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:08.054865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d5f1c-dc9f-440a-befd-302b449aaa61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:08.081817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0a1d5f1c-dc9f-440a-befd-302b449aaa61 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:08.083092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0a1d5f1c-dc9f-440a-befd-302b449aaa61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:08.112449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:08.123366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c071a2-2711-4b36-9953-83dcfce6258c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:08.149443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:08.163428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 136/277] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:08.237388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c071a2-2711-4b36-9953-83dcfce6258c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:08.274114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2bb62-d7d2-4ca1-88a2-33f7e4bdcda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:08.274789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2bb62-d7d2-4ca1-88a2-33f7e4bdcda1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:08.275108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2bb62-d7d2-4ca1-88a2-33f7e4bdcda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:08.275383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2bb62-d7d2-4ca1-88a2-33f7e4bdcda1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:08.286462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2bb62-d7d2-4ca1-88a2-33f7e4bdcda1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-aac2bb62-d7d2-4ca1-88a2-33f7e4bdcda1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:08.291448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:08.293705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:08.295698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 142/278] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:08.299192 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d5f1c-dc9f-440a-befd-302b449aaa61 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:08.306328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2bb62-d7d2-4ca1-88a2-33f7e4bdcda1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:08.412103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:08.431482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d5f1c-dc9f-440a-befd-302b449aaa61 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:08.431641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2bb62-d7d2-4ca1-88a2-33f7e4bdcda1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:08.464776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43149e-dd71-4276-835f-45549bfe5766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:08.465396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43149e-dd71-4276-835f-45549bfe5766 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:08.465669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43149e-dd71-4276-835f-45549bfe5766 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:08.465933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43149e-dd71-4276-835f-45549bfe5766 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:08.467020 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43149e-dd71-4276-835f-45549bfe5766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:08.467392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80229df3-198f-44cb-ad75-5eb37fbf4780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:08.468625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80229df3-198f-44cb-ad75-5eb37fbf4780 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:08.468903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80229df3-198f-44cb-ad75-5eb37fbf4780 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:08.469056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43149e-dd71-4276-835f-45549bfe5766 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:08.469159 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80229df3-198f-44cb-ad75-5eb37fbf4780 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:08.469875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80229df3-198f-44cb-ad75-5eb37fbf4780 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:08.470264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80229df3-198f-44cb-ad75-5eb37fbf4780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:08.475379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 143/279] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:08.475379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43149e-dd71-4276-835f-45549bfe5766 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:08.475687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80229df3-198f-44cb-ad75-5eb37fbf4780 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:08.482974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 137/280] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:08.483968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80229df3-198f-44cb-ad75-5eb37fbf4780 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:08.510674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:08.571392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43149e-dd71-4276-835f-45549bfe5766 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:08.585920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80229df3-198f-44cb-ad75-5eb37fbf4780 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:08.605417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260afe85-6d81-49c5-abed-65e21671fd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:08.606046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260afe85-6d81-49c5-abed-65e21671fd01 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:08.606322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260afe85-6d81-49c5-abed-65e21671fd01 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:08.606593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260afe85-6d81-49c5-abed-65e21671fd01 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:08.616332 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260afe85-6d81-49c5-abed-65e21671fd01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:08.619266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471512fc-97ea-4744-a429-cfdbe5fb8b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:08.619886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471512fc-97ea-4744-a429-cfdbe5fb8b33 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:08.620163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471512fc-97ea-4744-a429-cfdbe5fb8b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:08.620432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471512fc-97ea-4744-a429-cfdbe5fb8b33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-260afe85-6d81-49c5-abed-65e21671fd01 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:08.621183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:08.622157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:08.622947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:08.627119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 144/281] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:08.627978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260afe85-6d81-49c5-abed-65e21671fd01 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:08.630819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471512fc-97ea-4744-a429-cfdbe5fb8b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-471512fc-97ea-4744-a429-cfdbe5fb8b33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:08.636232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:08.637281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 138/282] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:08.638191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471512fc-97ea-4744-a429-cfdbe5fb8b33 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:08.641349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:08.726830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:08.734030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260afe85-6d81-49c5-abed-65e21671fd01 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:08.742176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471512fc-97ea-4744-a429-cfdbe5fb8b33 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:08.768590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5e1b4-8ae7-4bb4-828d-3cced7ac7ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:08.769183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5e1b4-8ae7-4bb4-828d-3cced7ac7ed9 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:08.769452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5e1b4-8ae7-4bb4-828d-3cced7ac7ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:08.769720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5e1b4-8ae7-4bb4-828d-3cced7ac7ed9 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:08.770357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5e1b4-8ae7-4bb4-828d-3cced7ac7ed9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:08.771821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5e1b4-8ae7-4bb4-828d-3cced7ac7ed9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:08.777075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2f5e9-532a-4138-b26f-8fdb24ec0d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:08.777704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2f5e9-532a-4138-b26f-8fdb24ec0d79 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:08.777982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2f5e9-532a-4138-b26f-8fdb24ec0d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:08.778251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2f5e9-532a-4138-b26f-8fdb24ec0d79 None admin] PATH_INFO: `/v3/users/610093d361144252b28a554487910ddb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:08.792656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2f5e9-532a-4138-b26f-8fdb24ec0d79 None admin] RBAC: Authorizing `identity:get_user(user_id=610093d361144252b28a554487910ddb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:08.792908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 145/283] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:48:08 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:48:08.793725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5e1b4-8ae7-4bb4-828d-3cced7ac7ed9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:08.794468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2f5e9-532a-4138-b26f-8fdb24ec0d79 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:08.801451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 139/284] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:08 2022] GET /identity/v3/users/610093d361144252b28a554487910ddb => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:08.801451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2f5e9-532a-4138-b26f-8fdb24ec0d79 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:08.872439 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:08.927068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5e1b4-8ae7-4bb4-828d-3cced7ac7ed9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:08.943207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2f5e9-532a-4138-b26f-8fdb24ec0d79 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:08.974832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca97af5-dcb0-45ed-9c2f-bb3d46f043fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:08.975739 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca97af5-dcb0-45ed-9c2f-bb3d46f043fa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:08.976071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca97af5-dcb0-45ed-9c2f-bb3d46f043fa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:08.976292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca97af5-dcb0-45ed-9c2f-bb3d46f043fa None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:08.976965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca97af5-dcb0-45ed-9c2f-bb3d46f043fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:08.977273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca97af5-dcb0-45ed-9c2f-bb3d46f043fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:08.978674 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca97af5-dcb0-45ed-9c2f-bb3d46f043fa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:08.980199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f853744-1479-4e0b-8773-c1ffeb7907de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:08.981281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f853744-1479-4e0b-8773-c1ffeb7907de None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:08.981281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f853744-1479-4e0b-8773-c1ffeb7907de None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:08.981459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f853744-1479-4e0b-8773-c1ffeb7907de None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:08.981996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f853744-1479-4e0b-8773-c1ffeb7907de None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:08.982299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f853744-1479-4e0b-8773-c1ffeb7907de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:08.983728 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f853744-1479-4e0b-8773-c1ffeb7907de None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:08.988895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 146/285] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:08.990836 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca97af5-dcb0-45ed-9c2f-bb3d46f043fa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:08.992668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 140/286] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:08.997910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f853744-1479-4e0b-8773-c1ffeb7907de None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:09.008559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:09.117866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca97af5-dcb0-45ed-9c2f-bb3d46f043fa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:09.131060 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f853744-1479-4e0b-8773-c1ffeb7907de None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:09.153676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:09.164287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0405a0-bbdc-4c58-91f4-94ac1f99d85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:09.164915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0405a0-bbdc-4c58-91f4-94ac1f99d85d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:09.165205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0405a0-bbdc-4c58-91f4-94ac1f99d85d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:09.165483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0405a0-bbdc-4c58-91f4-94ac1f99d85d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:09.166560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0405a0-bbdc-4c58-91f4-94ac1f99d85d None admin] RBAC: Adding query filter params (role.id=a0db5bd603fc4096917bbcd8c6932bc0, user.id=610093d361144252b28a554487910ddb, scope.system=all) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:09.167216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0405a0-bbdc-4c58-91f4-94ac1f99d85d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:09.168846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0405a0-bbdc-4c58-91f4-94ac1f99d85d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:09.175233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 147/287] 10.222.0.86 () {56 vars in 1308 bytes} [Mon Aug 29 18:48:08 2022] GET /identity/v3/role_assignments?user.id=610093d361144252b28a554487910ddb&scope.system=all&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:09.175689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3a1aa-4068-43ab-91a4-2c356920b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:09.176053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0405a0-bbdc-4c58-91f4-94ac1f99d85d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:09.176815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a1aa-4068-43ab-91a4-2c356920b2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:09.177293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a1aa-4068-43ab-91a4-2c356920b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:09.177773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a1aa-4068-43ab-91a4-2c356920b2a6 None admin] PATH_INFO: `/v3/users/cc93504afac5417ab0e7e60a1c5be6c5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:09.183326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3a1aa-4068-43ab-91a4-2c356920b2a6 None admin] RBAC: Authorizing `identity:get_user(user_id=cc93504afac5417ab0e7e60a1c5be6c5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:09.183326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3a1aa-4068-43ab-91a4-2c356920b2a6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:09.184248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 141/288] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:08 2022] GET /identity/v3/users/cc93504afac5417ab0e7e60a1c5be6c5 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:09.192410 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3a1aa-4068-43ab-91a4-2c356920b2a6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:09.369478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:09.399813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0405a0-bbdc-4c58-91f4-94ac1f99d85d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:09.413563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3a1aa-4068-43ab-91a4-2c356920b2a6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:09.462128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072ff57-8fad-48a5-b88a-8aa46d3aebab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:09.462912 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072ff57-8fad-48a5-b88a-8aa46d3aebab None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:09.463224 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072ff57-8fad-48a5-b88a-8aa46d3aebab None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:09.463508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072ff57-8fad-48a5-b88a-8aa46d3aebab None admin] PATH_INFO: `/v3/users/3b29f427ad814d69a7cdc5c8329db736` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:09.464840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072ff57-8fad-48a5-b88a-8aa46d3aebab None admin] RBAC: Authorizing `identity:get_user(user_id=3b29f427ad814d69a7cdc5c8329db736)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:09.466695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072ff57-8fad-48a5-b88a-8aa46d3aebab None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:09.469389 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 148/289] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:09 2022] GET /identity/v3/users/3b29f427ad814d69a7cdc5c8329db736 => generated 297 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:09.474969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072ff57-8fad-48a5-b88a-8aa46d3aebab None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:09.481915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cdbe4-ede4-43ef-96d0-702430a1c509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:09.482743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cdbe4-ede4-43ef-96d0-702430a1c509 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:09.483755 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cdbe4-ede4-43ef-96d0-702430a1c509 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:09.484174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cdbe4-ede4-43ef-96d0-702430a1c509 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:09.485605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cdbe4-ede4-43ef-96d0-702430a1c509 None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:09.495459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cdbe4-ede4-43ef-96d0-702430a1c509 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:09.497483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 142/290] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:09 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:09.505133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cdbe4-ede4-43ef-96d0-702430a1c509 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:09.548037 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:09.595507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072ff57-8fad-48a5-b88a-8aa46d3aebab None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:09.632596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf9bf5-f91f-404c-ab4e-d4733087592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:09.633325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9bf5-f91f-404c-ab4e-d4733087592b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:09.633682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9bf5-f91f-404c-ab4e-d4733087592b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:09.634031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9bf5-f91f-404c-ab4e-d4733087592b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:09.656624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cdbe4-ede4-43ef-96d0-702430a1c509 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:09.713948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb5731-a3d2-4f70-a774-7da09894f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:09.714626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb5731-a3d2-4f70-a774-7da09894f61e None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:09.714909 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb5731-a3d2-4f70-a774-7da09894f61e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:09.715205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb5731-a3d2-4f70-a774-7da09894f61e None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db/users/cc93504afac5417ab0e7e60a1c5be6c5/roles/4b69aa55fbbf4624b0b6036a5a8b2d4d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:09.717957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb5731-a3d2-4f70-a774-7da09894f61e None admin] RBAC: Authorizing `identity:create_grant(project_id=f456bff4d42f4ff194d2ac607af555db, user_id=cc93504afac5417ab0e7e60a1c5be6c5, role_id=4b69aa55fbbf4624b0b6036a5a8b2d4d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:09.719621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb5731-a3d2-4f70-a774-7da09894f61e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:09.738061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 143/291] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:48:09 2022] PUT /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db/users/cc93504afac5417ab0e7e60a1c5be6c5/roles/4b69aa55fbbf4624b0b6036a5a8b2d4d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:48:09.748032 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf9bf5-f91f-404c-ab4e-d4733087592b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:09.750043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf9bf5-f91f-404c-ab4e-d4733087592b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:09.872059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:09.925769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 149/292] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:48:09.931268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb5731-a3d2-4f70-a774-7da09894f61e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:09.934170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:10.039829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb5731-a3d2-4f70-a774-7da09894f61e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:10.081768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2d468-6e32-4f26-b1d0-941a3d2422aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:10.082435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d468-6e32-4f26-b1d0-941a3d2422aa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:10.082715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d468-6e32-4f26-b1d0-941a3d2422aa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:10.082990 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d468-6e32-4f26-b1d0-941a3d2422aa None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:10.084246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2d468-6e32-4f26-b1d0-941a3d2422aa None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:10.085735 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2d468-6e32-4f26-b1d0-941a3d2422aa None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:10.087553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 144/293] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:09 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:10.093646 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf9bf5-f91f-404c-ab4e-d4733087592b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:10.216317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf9bf5-f91f-404c-ab4e-d4733087592b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:10.259616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0415e12-437d-4def-9cde-da923d2f10d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:10.260223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0415e12-437d-4def-9cde-da923d2f10d0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:10.260494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0415e12-437d-4def-9cde-da923d2f10d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:10.260753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0415e12-437d-4def-9cde-da923d2f10d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:10.261482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0415e12-437d-4def-9cde-da923d2f10d0 None admin] RBAC: Adding query filter params (role.id=a0db5bd603fc4096917bbcd8c6932bc0, scope.project.id=f456bff4d42f4ff194d2ac607af555db, user.id=3b29f427ad814d69a7cdc5c8329db736) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:10.261849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0415e12-437d-4def-9cde-da923d2f10d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:10.263306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0415e12-437d-4def-9cde-da923d2f10d0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:10.269853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 150/294] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:10 2022] GET /identity/v3/role_assignments?user.id=3b29f427ad814d69a7cdc5c8329db736&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:11.063858 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3b49-1c83-4834-8ba1-54cf4afdc86f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:11.064636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3b49-1c83-4834-8ba1-54cf4afdc86f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:11.064636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3b49-1c83-4834-8ba1-54cf4afdc86f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:11.065965 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 145/295] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:11.072704 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c0a8e-5c83-4b13-9cbe-f490d7709fce None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:11.073231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c0a8e-5c83-4b13-9cbe-f490d7709fce None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:11.073748 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c0a8e-5c83-4b13-9cbe-f490d7709fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:11.094186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-309c0a8e-5c83-4b13-9cbe-f490d7709fce None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:11.095567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-309c0a8e-5c83-4b13-9cbe-f490d7709fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:11.114204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:11.167686 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 151/296] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:11.298381 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1af60-db42-4dc5-a12e-363f35304f9e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:11.298659 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1af60-db42-4dc5-a12e-363f35304f9e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:11.298943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1af60-db42-4dc5-a12e-363f35304f9e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:11.300311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 146/297] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:11.305696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e427b46-ed5d-41da-b175-412d8c6dfdc8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:11.306142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e427b46-ed5d-41da-b175-412d8c6dfdc8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:11.306327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e427b46-ed5d-41da-b175-412d8c6dfdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:11.325055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5e427b46-ed5d-41da-b175-412d8c6dfdc8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:11.326179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e427b46-ed5d-41da-b175-412d8c6dfdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:11.377088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64608516-0e3c-448f-bdbb-73a828272756 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:11.377608 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64608516-0e3c-448f-bdbb-73a828272756 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:11.377889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64608516-0e3c-448f-bdbb-73a828272756 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:11.379245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 147/298] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:48:11.384316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae9f32-2929-4010-842f-8b3a57265dd7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:11.384602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae9f32-2929-4010-842f-8b3a57265dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:11.384880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae9f32-2929-4010-842f-8b3a57265dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:11.407367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 152/299] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:11.408464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f5ae9f32-2929-4010-842f-8b3a57265dd7 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:11.410488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f5ae9f32-2929-4010-842f-8b3a57265dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:11.456613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e427b46-ed5d-41da-b175-412d8c6dfdc8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:11.478879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:11.484184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 148/300] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:11.547328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:11.566323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e427b46-ed5d-41da-b175-412d8c6dfdc8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:11.602951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074147f-1299-4e97-a7b5-49c12adc82fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:11.603917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074147f-1299-4e97-a7b5-49c12adc82fd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:11.604398 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074147f-1299-4e97-a7b5-49c12adc82fd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:11.604890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074147f-1299-4e97-a7b5-49c12adc82fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:11.615592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074147f-1299-4e97-a7b5-49c12adc82fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2074147f-1299-4e97-a7b5-49c12adc82fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:11.620811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:11.621671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:11.622463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:11.625066 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 153/301] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:11.630963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae9f32-2929-4010-842f-8b3a57265dd7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:11.682399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074147f-1299-4e97-a7b5-49c12adc82fd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:11.751288 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae9f32-2929-4010-842f-8b3a57265dd7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:11.788602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a41ca-7ea9-4cf4-8a82-14b9c285935e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:11.789523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a41ca-7ea9-4cf4-8a82-14b9c285935e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:11.789976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a41ca-7ea9-4cf4-8a82-14b9c285935e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:11.790436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a41ca-7ea9-4cf4-8a82-14b9c285935e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:11.791320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a41ca-7ea9-4cf4-8a82-14b9c285935e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:11.791622 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a41ca-7ea9-4cf4-8a82-14b9c285935e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:11.792998 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a41ca-7ea9-4cf4-8a82-14b9c285935e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:11.801710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 149/302] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:11.806937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a41ca-7ea9-4cf4-8a82-14b9c285935e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:11.810596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074147f-1299-4e97-a7b5-49c12adc82fd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:11.845528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99258159-11d8-4428-81af-e9e1bb0b0409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:11.846319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99258159-11d8-4428-81af-e9e1bb0b0409 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:11.846620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99258159-11d8-4428-81af-e9e1bb0b0409 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:11.846879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99258159-11d8-4428-81af-e9e1bb0b0409 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:11.848032 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99258159-11d8-4428-81af-e9e1bb0b0409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:11.849514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99258159-11d8-4428-81af-e9e1bb0b0409 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:11.855535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 154/303] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:11.855535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99258159-11d8-4428-81af-e9e1bb0b0409 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:11.865607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:11.878295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:11.921208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a41ca-7ea9-4cf4-8a82-14b9c285935e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:11.957925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60651e-4c9d-48b7-b65f-72abeaccc968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:11.959039 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60651e-4c9d-48b7-b65f-72abeaccc968 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:11.959461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60651e-4c9d-48b7-b65f-72abeaccc968 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:11.960008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60651e-4c9d-48b7-b65f-72abeaccc968 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:11.960401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99258159-11d8-4428-81af-e9e1bb0b0409 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:11.972759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60651e-4c9d-48b7-b65f-72abeaccc968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8c60651e-4c9d-48b7-b65f-72abeaccc968 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:11.979514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:11.980583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:11.981559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:11.984188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 150/304] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:11.985297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60651e-4c9d-48b7-b65f-72abeaccc968 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:11.996546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db9836-a46a-44b0-891f-71b383eca745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:11.997179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db9836-a46a-44b0-891f-71b383eca745 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:11.997460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db9836-a46a-44b0-891f-71b383eca745 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:11.997734 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db9836-a46a-44b0-891f-71b383eca745 None admin] PATH_INFO: `/v3/users/7f993e0d69314fac8ed65a232e07e28d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:12.012321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db9836-a46a-44b0-891f-71b383eca745 None admin] RBAC: Authorizing `identity:get_user(user_id=7f993e0d69314fac8ed65a232e07e28d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:12.014132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db9836-a46a-44b0-891f-71b383eca745 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:12.016450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 155/305] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:11 2022] GET /identity/v3/users/7f993e0d69314fac8ed65a232e07e28d => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:12.017244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db9836-a46a-44b0-891f-71b383eca745 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:12.087339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:12.101238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60651e-4c9d-48b7-b65f-72abeaccc968 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:12.121223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db9836-a46a-44b0-891f-71b383eca745 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:12.125656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:12.137590 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ee7c3-277a-4c1d-8703-e374500f969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:12.138192 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ee7c3-277a-4c1d-8703-e374500f969d None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:12.138468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ee7c3-277a-4c1d-8703-e374500f969d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:12.138738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ee7c3-277a-4c1d-8703-e374500f969d None admin] PATH_INFO: `/v3/groups` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:12.139411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ee7c3-277a-4c1d-8703-e374500f969d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:12.140817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ee7c3-277a-4c1d-8703-e374500f969d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:12.155330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da667b2-0b66-4c5b-8574-c4dfaecdf9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:12.155883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 151/306] 10.222.0.86 () {60 vars in 1134 bytes} [Mon Aug 29 18:48:11 2022] POST /identity/v3/groups => generated 229 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:48:12.155986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da667b2-0b66-4c5b-8574-c4dfaecdf9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:12.156193 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da667b2-0b66-4c5b-8574-c4dfaecdf9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:12.156461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da667b2-0b66-4c5b-8574-c4dfaecdf9f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:12.156685 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ee7c3-277a-4c1d-8703-e374500f969d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:12.157131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da667b2-0b66-4c5b-8574-c4dfaecdf9f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:12.157420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da667b2-0b66-4c5b-8574-c4dfaecdf9f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:12.158778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da667b2-0b66-4c5b-8574-c4dfaecdf9f0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:12.179471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 156/307] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:12.181218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397f4bf-d4b3-4cf7-b295-31561523a1b0 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:12.181755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397f4bf-d4b3-4cf7-b295-31561523a1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:12.182037 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397f4bf-d4b3-4cf7-b295-31561523a1b0 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:12.183618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 157/308] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:12.188924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397f4bf-d4b3-4cf7-b295-31561523a1b0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:12.279547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ee7c3-277a-4c1d-8703-e374500f969d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:12.304229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397f4bf-d4b3-4cf7-b295-31561523a1b0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:12.321208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8ba71-4a47-49aa-ad60-436ac557f685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:12.322462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ba71-4a47-49aa-ad60-436ac557f685 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:12.322462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ba71-4a47-49aa-ad60-436ac557f685 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:12.322462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ba71-4a47-49aa-ad60-436ac557f685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:12.323082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8ba71-4a47-49aa-ad60-436ac557f685 None admin] RBAC: Adding query filter params (role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d, user.id=7f993e0d69314fac8ed65a232e07e28d, scope.system=all) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:12.323392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8ba71-4a47-49aa-ad60-436ac557f685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:12.325386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8ba71-4a47-49aa-ad60-436ac557f685 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:12.331221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 152/309] 10.222.0.86 () {56 vars in 1308 bytes} [Mon Aug 29 18:48:12 2022] GET /identity/v3/role_assignments?user.id=7f993e0d69314fac8ed65a232e07e28d&scope.system=all&role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:12.332404 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5b2d8-8a82-4df3-b049-908a1c7f7d3e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:12.332684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5b2d8-8a82-4df3-b049-908a1c7f7d3e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:12.332955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5b2d8-8a82-4df3-b049-908a1c7f7d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:12.352648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:12.354264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b2c5b2d8-8a82-4df3-b049-908a1c7f7d3e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:12.355516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b2c5b2d8-8a82-4df3-b049-908a1c7f7d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:12.363219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f804e23-029d-4769-a42b-ff21aa3acfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:12.364261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f804e23-029d-4769-a42b-ff21aa3acfd2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:12.364739 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f804e23-029d-4769-a42b-ff21aa3acfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:12.365958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f804e23-029d-4769-a42b-ff21aa3acfd2 None admin] PATH_INFO: `/v3/users/610093d361144252b28a554487910ddb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:12.367516 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f804e23-029d-4769-a42b-ff21aa3acfd2 None admin] RBAC: Authorizing `identity:get_user(user_id=610093d361144252b28a554487910ddb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:12.370411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f804e23-029d-4769-a42b-ff21aa3acfd2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:12.373140 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 158/310] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:12 2022] GET /identity/v3/users/610093d361144252b28a554487910ddb => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:12.382960 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f804e23-029d-4769-a42b-ff21aa3acfd2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:12.411054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:12.458026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 153/311] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:12.516300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f804e23-029d-4769-a42b-ff21aa3acfd2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:12.551557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ae5d4-2b45-48fd-ab0f-0f404cf6127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:12.552205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ae5d4-2b45-48fd-ab0f-0f404cf6127a None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:12.552480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ae5d4-2b45-48fd-ab0f-0f404cf6127a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:12.552749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ae5d4-2b45-48fd-ab0f-0f404cf6127a None admin] PATH_INFO: `/v3/system/users/610093d361144252b28a554487910ddb/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:12.554147 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ae5d4-2b45-48fd-ab0f-0f404cf6127a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=610093d361144252b28a554487910ddb, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:12.555537 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ae5d4-2b45-48fd-ab0f-0f404cf6127a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:12.565219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 159/312] 10.222.0.86 () {58 vars in 1334 bytes} [Mon Aug 29 18:48:12 2022] PUT /identity/v3/system/users/610093d361144252b28a554487910ddb/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:48:12.736881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5b2d8-8a82-4df3-b049-908a1c7f7d3e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:12.849243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5b2d8-8a82-4df3-b049-908a1c7f7d3e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:12.886291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b0d1e-28e9-48d0-8d1a-835bedd4b3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:12.886989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b0d1e-28e9-48d0-8d1a-835bedd4b3ad None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:12.887297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b0d1e-28e9-48d0-8d1a-835bedd4b3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:12.887565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b0d1e-28e9-48d0-8d1a-835bedd4b3ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:12.898205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b0d1e-28e9-48d0-8d1a-835bedd4b3ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-727b0d1e-28e9-48d0-8d1a-835bedd4b3ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:12.903069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:12.903925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:12.905033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 154/313] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:12.911815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ae5d4-2b45-48fd-ab0f-0f404cf6127a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:12.973919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:13.015788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ae5d4-2b45-48fd-ab0f-0f404cf6127a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:13.053365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1fb59-fecc-48bf-8ce5-56fb70d278a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:13.053973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1fb59-fecc-48bf-8ce5-56fb70d278a6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:13.054252 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1fb59-fecc-48bf-8ce5-56fb70d278a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:13.054523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1fb59-fecc-48bf-8ce5-56fb70d278a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:13.055210 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1fb59-fecc-48bf-8ce5-56fb70d278a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:13.055509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1fb59-fecc-48bf-8ce5-56fb70d278a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:13.056861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1fb59-fecc-48bf-8ce5-56fb70d278a6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:13.065326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 160/314] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:13.071050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b0d1e-28e9-48d0-8d1a-835bedd4b3ad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:13.112522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:13.181911 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b0d1e-28e9-48d0-8d1a-835bedd4b3ad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:13.218944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cd1b8-c902-4c7a-8ae2-69104638d881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:13.219605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cd1b8-c902-4c7a-8ae2-69104638d881 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:13.219889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cd1b8-c902-4c7a-8ae2-69104638d881 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:13.220154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cd1b8-c902-4c7a-8ae2-69104638d881 None admin] PATH_INFO: `/v3/users/cc93504afac5417ab0e7e60a1c5be6c5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:13.221309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cd1b8-c902-4c7a-8ae2-69104638d881 None admin] RBAC: Authorizing `identity:get_user(user_id=cc93504afac5417ab0e7e60a1c5be6c5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:13.222762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cd1b8-c902-4c7a-8ae2-69104638d881 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:13.224430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 155/315] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:13 2022] GET /identity/v3/users/cc93504afac5417ab0e7e60a1c5be6c5 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:13.230026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1fb59-fecc-48bf-8ce5-56fb70d278a6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:13.341108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1fb59-fecc-48bf-8ce5-56fb70d278a6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:13.364897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:13.377001 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d976d-1d21-4d81-bc9c-458f2f3d9609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:13.377613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d976d-1d21-4d81-bc9c-458f2f3d9609 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:13.377892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d976d-1d21-4d81-bc9c-458f2f3d9609 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:13.378158 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d976d-1d21-4d81-bc9c-458f2f3d9609 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:13.449131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d976d-1d21-4d81-bc9c-458f2f3d9609 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:13.450689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d976d-1d21-4d81-bc9c-458f2f3d9609 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:13.555255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 161/316] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:13.560451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cd1b8-c902-4c7a-8ae2-69104638d881 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:13.667395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cd1b8-c902-4c7a-8ae2-69104638d881 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:13.669727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:13.710457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a63b9-cc59-4601-99c9-e40f970d4675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:13.711129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a63b9-cc59-4601-99c9-e40f970d4675 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:13.711412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a63b9-cc59-4601-99c9-e40f970d4675 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:13.711688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a63b9-cc59-4601-99c9-e40f970d4675 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:13.712863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a63b9-cc59-4601-99c9-e40f970d4675 None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:13.714314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a63b9-cc59-4601-99c9-e40f970d4675 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:13.716284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 156/317] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:13 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:13.722788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d976d-1d21-4d81-bc9c-458f2f3d9609 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:13.747872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:13.831915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d976d-1d21-4d81-bc9c-458f2f3d9609 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:13.867973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa0769-e86d-40e4-af27-0c2de18b2735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:13.868592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa0769-e86d-40e4-af27-0c2de18b2735 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:13.868877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa0769-e86d-40e4-af27-0c2de18b2735 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:13.869153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa0769-e86d-40e4-af27-0c2de18b2735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:13.869885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa0769-e86d-40e4-af27-0c2de18b2735 None admin] RBAC: Adding query filter params (role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d, scope.project.id=f456bff4d42f4ff194d2ac607af555db, user.id=cc93504afac5417ab0e7e60a1c5be6c5) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:13.870198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa0769-e86d-40e4-af27-0c2de18b2735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:13.871675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa0769-e86d-40e4-af27-0c2de18b2735 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:13.878285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 162/318] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:13 2022] GET /identity/v3/role_assignments?user.id=cc93504afac5417ab0e7e60a1c5be6c5&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:14.195630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68720f38-dea0-41c0-af3b-e3207a763eff None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:14.196112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68720f38-dea0-41c0-af3b-e3207a763eff None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:14.196112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68720f38-dea0-41c0-af3b-e3207a763eff None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:14.197720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 157/319] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:14.203693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e802fc-5377-495f-bf16-d9e92155f3fe None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:14.203981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e802fc-5377-495f-bf16-d9e92155f3fe None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:14.204253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e802fc-5377-495f-bf16-d9e92155f3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:14.223237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e7e802fc-5377-495f-bf16-d9e92155f3fe None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:14.224431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e7e802fc-5377-495f-bf16-d9e92155f3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:14.293360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 163/320] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:14.341905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2c61f-577a-4e9c-b00d-df372f006005 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:14.342238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2c61f-577a-4e9c-b00d-df372f006005 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:14.342487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2c61f-577a-4e9c-b00d-df372f006005 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:14.343975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 158/321] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:14.348901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda5e47-beac-428a-b5a4-f965491accb1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:14.349187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda5e47-beac-428a-b5a4-f965491accb1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:14.349459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda5e47-beac-428a-b5a4-f965491accb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:14.367716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2eda5e47-beac-428a-b5a4-f965491accb1 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:14.368946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2eda5e47-beac-428a-b5a4-f965491accb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:14.383069 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:14.442969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 164/322] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:14.551953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a3e02-f854-492f-9261-76269beab2f3 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:14.552279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a3e02-f854-492f-9261-76269beab2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:14.552511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a3e02-f854-492f-9261-76269beab2f3 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:14.553980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 159/323] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:14.559073 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c6ec2-d936-4b39-94b3-0d6cb44da5dd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:14.559368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c6ec2-d936-4b39-94b3-0d6cb44da5dd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:14.559643 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c6ec2-d936-4b39-94b3-0d6cb44da5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:14.570519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a3e02-f854-492f-9261-76269beab2f3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:14.578528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4f6c6ec2-d936-4b39-94b3-0d6cb44da5dd None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:14.579814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4f6c6ec2-d936-4b39-94b3-0d6cb44da5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:14.650818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 165/324] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:14.691504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a3e02-f854-492f-9261-76269beab2f3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:14.707300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c6ec2-d936-4b39-94b3-0d6cb44da5dd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:14.730104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead9405-3abb-40be-be42-cc2d3dfd7f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:14.730939 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead9405-3abb-40be-be42-cc2d3dfd7f33 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:14.731244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead9405-3abb-40be-be42-cc2d3dfd7f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:14.731513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead9405-3abb-40be-be42-cc2d3dfd7f33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:14.732588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead9405-3abb-40be-be42-cc2d3dfd7f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:14.734026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead9405-3abb-40be-be42-cc2d3dfd7f33 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:14.735746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 160/325] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:14.741229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead9405-3abb-40be-be42-cc2d3dfd7f33 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:14.783814 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:14.794684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:14.819266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c6ec2-d936-4b39-94b3-0d6cb44da5dd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:14.848292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead9405-3abb-40be-be42-cc2d3dfd7f33 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:14.854351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea0d12-9b0c-4eca-8211-5a4462cb4740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:14.854988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea0d12-9b0c-4eca-8211-5a4462cb4740 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:14.855304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea0d12-9b0c-4eca-8211-5a4462cb4740 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:14.855992 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea0d12-9b0c-4eca-8211-5a4462cb4740 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:14.867226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea0d12-9b0c-4eca-8211-5a4462cb4740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1cea0d12-9b0c-4eca-8211-5a4462cb4740 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:14.872033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:14.872903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:14.874243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 166/326] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:14.879361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea0d12-9b0c-4eca-8211-5a4462cb4740 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:14.889992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61681e09-4da7-4875-9050-c95cac4fbf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:14.890923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61681e09-4da7-4875-9050-c95cac4fbf74 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:14.891426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61681e09-4da7-4875-9050-c95cac4fbf74 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:14.892096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61681e09-4da7-4875-9050-c95cac4fbf74 None admin] PATH_INFO: `/v3/groups` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:14.892884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61681e09-4da7-4875-9050-c95cac4fbf74 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:14.894577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61681e09-4da7-4875-9050-c95cac4fbf74 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:14.911250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 161/327] 10.222.0.86 () {60 vars in 1134 bytes} [Mon Aug 29 18:48:14 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:48:14.928723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61681e09-4da7-4875-9050-c95cac4fbf74 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:15.017163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea0d12-9b0c-4eca-8211-5a4462cb4740 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:15.036949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:15.040646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61681e09-4da7-4875-9050-c95cac4fbf74 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:15.059325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157d360-b9b5-4e30-988f-7633fd98d1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:15.059941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157d360-b9b5-4e30-988f-7633fd98d1ff None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:15.061492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157d360-b9b5-4e30-988f-7633fd98d1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:15.061492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157d360-b9b5-4e30-988f-7633fd98d1ff None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:15.061680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157d360-b9b5-4e30-988f-7633fd98d1ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:15.061961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157d360-b9b5-4e30-988f-7633fd98d1ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:15.063350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157d360-b9b5-4e30-988f-7633fd98d1ff None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:15.071627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:15.078754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98269f4a-e657-4f4e-9663-d7ce996e41a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:15.079686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98269f4a-e657-4f4e-9663-d7ce996e41a7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:15.080137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98269f4a-e657-4f4e-9663-d7ce996e41a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:15.080347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 167/328] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:15.080794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98269f4a-e657-4f4e-9663-d7ce996e41a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:15.082229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157d360-b9b5-4e30-988f-7633fd98d1ff None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:15.093800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98269f4a-e657-4f4e-9663-d7ce996e41a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-98269f4a-e657-4f4e-9663-d7ce996e41a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:15.099187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:15.100070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:15.100866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 162/329] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:15.107166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98269f4a-e657-4f4e-9663-d7ce996e41a7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:15.224954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157d360-b9b5-4e30-988f-7633fd98d1ff None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:15.239213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98269f4a-e657-4f4e-9663-d7ce996e41a7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:15.269217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462e0c7-40bb-4f5f-bdc9-5db284f8fc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:15.270360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462e0c7-40bb-4f5f-bdc9-5db284f8fc1d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:15.270780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462e0c7-40bb-4f5f-bdc9-5db284f8fc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:15.271713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462e0c7-40bb-4f5f-bdc9-5db284f8fc1d None admin] PATH_INFO: `/v3/users/7f993e0d69314fac8ed65a232e07e28d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:15.272558 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462e0c7-40bb-4f5f-bdc9-5db284f8fc1d None admin] RBAC: Authorizing `identity:get_user(user_id=7f993e0d69314fac8ed65a232e07e28d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:15.274952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462e0c7-40bb-4f5f-bdc9-5db284f8fc1d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:15.277011 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 168/330] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:15 2022] GET /identity/v3/users/7f993e0d69314fac8ed65a232e07e28d => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:15.282029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2741184-0695-4855-9524-48b39586182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:15.282029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2741184-0695-4855-9524-48b39586182f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:15.282029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2741184-0695-4855-9524-48b39586182f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:15.282029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2741184-0695-4855-9524-48b39586182f None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:15.282704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2741184-0695-4855-9524-48b39586182f None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:15.283031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2741184-0695-4855-9524-48b39586182f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:15.284409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2741184-0695-4855-9524-48b39586182f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:15.284892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462e0c7-40bb-4f5f-bdc9-5db284f8fc1d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:15.294862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 163/331] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:15.302822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2741184-0695-4855-9524-48b39586182f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:15.337271 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:15.365050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:15.410017 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462e0c7-40bb-4f5f-bdc9-5db284f8fc1d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:15.444517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2741184-0695-4855-9524-48b39586182f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:15.465035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca45a5d-dbeb-4ca8-8b6e-3829985b82e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:15.465990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca45a5d-dbeb-4ca8-8b6e-3829985b82e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:15.466446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca45a5d-dbeb-4ca8-8b6e-3829985b82e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:15.466896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca45a5d-dbeb-4ca8-8b6e-3829985b82e9 None admin] PATH_INFO: `/v3/system/users/7f993e0d69314fac8ed65a232e07e28d/roles/4b69aa55fbbf4624b0b6036a5a8b2d4d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:15.469903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca45a5d-dbeb-4ca8-8b6e-3829985b82e9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7f993e0d69314fac8ed65a232e07e28d, role_id=4b69aa55fbbf4624b0b6036a5a8b2d4d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:15.472204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca45a5d-dbeb-4ca8-8b6e-3829985b82e9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:15.482627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 169/332] 10.222.0.86 () {58 vars in 1334 bytes} [Mon Aug 29 18:48:15 2022] PUT /identity/v3/system/users/7f993e0d69314fac8ed65a232e07e28d/roles/4b69aa55fbbf4624b0b6036a5a8b2d4d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:48:15.487209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93f921-1f1d-43e9-9ee6-8c1476c650f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:15.487209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93f921-1f1d-43e9-9ee6-8c1476c650f8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:15.487209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93f921-1f1d-43e9-9ee6-8c1476c650f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:15.487503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93f921-1f1d-43e9-9ee6-8c1476c650f8 None admin] PATH_INFO: `/v3/users/610093d361144252b28a554487910ddb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:15.490567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93f921-1f1d-43e9-9ee6-8c1476c650f8 None admin] RBAC: Authorizing `identity:get_user(user_id=610093d361144252b28a554487910ddb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:15.492263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93f921-1f1d-43e9-9ee6-8c1476c650f8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:15.494413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 164/333] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:15 2022] GET /identity/v3/users/610093d361144252b28a554487910ddb => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:15.501169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca45a5d-dbeb-4ca8-8b6e-3829985b82e9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:15.624279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca45a5d-dbeb-4ca8-8b6e-3829985b82e9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:15.657589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:15.662897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabf558-c5f8-480e-954f-970596d2d31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:15.663590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabf558-c5f8-480e-954f-970596d2d31d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:15.664081 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabf558-c5f8-480e-954f-970596d2d31d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:15.664418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabf558-c5f8-480e-954f-970596d2d31d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:15.665296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabf558-c5f8-480e-954f-970596d2d31d None admin] RBAC: Adding query filter params (role.id=a0db5bd603fc4096917bbcd8c6932bc0, user.id=610093d361144252b28a554487910ddb, scope.system=all) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:15.665668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabf558-c5f8-480e-954f-970596d2d31d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:15.667636 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabf558-c5f8-480e-954f-970596d2d31d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:15.685647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 170/334] 10.222.0.86 () {56 vars in 1308 bytes} [Mon Aug 29 18:48:15 2022] GET /identity/v3/role_assignments?user.id=610093d361144252b28a554487910ddb&scope.system=all&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:17.160852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16237b3e-8484-4de2-9a9b-377d2fdc4b36 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:17.161395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16237b3e-8484-4de2-9a9b-377d2fdc4b36 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:17.161395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16237b3e-8484-4de2-9a9b-377d2fdc4b36 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:17.162859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 165/335] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:17.168281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b8823-c526-49b7-8249-fea531d6e545 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:17.169091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b8823-c526-49b7-8249-fea531d6e545 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:17.169563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b8823-c526-49b7-8249-fea531d6e545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:17.193911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-aa6b8823-c526-49b7-8249-fea531d6e545 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:17.195476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-aa6b8823-c526-49b7-8249-fea531d6e545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:17.284404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 171/336] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:17.819500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1b3e-fe40-4f1b-a015-b762aa3bfd01 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:17.819780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1b3e-fe40-4f1b-a015-b762aa3bfd01 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:17.820360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1b3e-fe40-4f1b-a015-b762aa3bfd01 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:17.822193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 166/337] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:17.824727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b8823-c526-49b7-8249-fea531d6e545 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:17.827605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d9f9-952b-4d25-a519-07a5cf7cc1ea None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:17.828107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d9f9-952b-4d25-a519-07a5cf7cc1ea None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:17.828576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d9f9-952b-4d25-a519-07a5cf7cc1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:17.847854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3fe3d9f9-952b-4d25-a519-07a5cf7cc1ea None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:17.849478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3fe3d9f9-952b-4d25-a519-07a5cf7cc1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:17.938288 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 167/338] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:17.940055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce1b9-adec-42e4-a532-a7ff5b60eebc None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:17.940764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce1b9-adec-42e4-a532-a7ff5b60eebc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:17.941238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce1b9-adec-42e4-a532-a7ff5b60eebc None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:17.942847 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 168/339] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:17.953013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7887d-c6f9-4eae-b060-361757ecccde None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:17.953585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7887d-c6f9-4eae-b060-361757ecccde None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:17.954148 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7887d-c6f9-4eae-b060-361757ecccde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:17.983743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3cb7887d-c6f9-4eae-b060-361757ecccde None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:17.985506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3cb7887d-c6f9-4eae-b060-361757ecccde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:18.006277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:18.039031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b8823-c526-49b7-8249-fea531d6e545 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:18.081071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:18.085720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 169/340] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:18.087243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a553631-6278-467d-b29a-b3faa4a73dde None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:18.087775 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a553631-6278-467d-b29a-b3faa4a73dde None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:18.088268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a553631-6278-467d-b29a-b3faa4a73dde None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:18.089716 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 170/341] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:18.095903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf032e3-63f8-40a8-87b3-7cf72f9e173a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:18.096742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf032e3-63f8-40a8-87b3-7cf72f9e173a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:18.097243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf032e3-63f8-40a8-87b3-7cf72f9e173a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:18.127298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904fa92-3182-431b-be85-92be1c6188f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:18.128375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904fa92-3182-431b-be85-92be1c6188f7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:18.128849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904fa92-3182-431b-be85-92be1c6188f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:18.129310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904fa92-3182-431b-be85-92be1c6188f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:18.132006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9bf032e3-63f8-40a8-87b3-7cf72f9e173a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:18.133230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9bf032e3-63f8-40a8-87b3-7cf72f9e173a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:18.155888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904fa92-3182-431b-be85-92be1c6188f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e904fa92-3182-431b-be85-92be1c6188f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:18.161504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:18.162388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:18.163188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:18.165725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 172/342] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:18.174454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904fa92-3182-431b-be85-92be1c6188f7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:18.235327 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 171/343] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:18.331194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904fa92-3182-431b-be85-92be1c6188f7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:18.374980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e1f73-49e0-4c51-bbad-125384652456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:18.375607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e1f73-49e0-4c51-bbad-125384652456 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:18.375884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e1f73-49e0-4c51-bbad-125384652456 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:18.376153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e1f73-49e0-4c51-bbad-125384652456 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:18.376815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e1f73-49e0-4c51-bbad-125384652456 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:18.377111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e1f73-49e0-4c51-bbad-125384652456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:18.378477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e1f73-49e0-4c51-bbad-125384652456 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:18.387157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 173/344] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:18.393672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf032e3-63f8-40a8-87b3-7cf72f9e173a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:18.414056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e1f73-49e0-4c51-bbad-125384652456 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:18.469802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:18.524675 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:18.533428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e1f73-49e0-4c51-bbad-125384652456 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:18.559587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf032e3-63f8-40a8-87b3-7cf72f9e173a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:18.570337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846faa3-5eaf-4df6-b5ca-303af8da9698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:18.571057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846faa3-5eaf-4df6-b5ca-303af8da9698 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:18.571344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846faa3-5eaf-4df6-b5ca-303af8da9698 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:18.571622 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846faa3-5eaf-4df6-b5ca-303af8da9698 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:18.591518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846faa3-5eaf-4df6-b5ca-303af8da9698 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6846faa3-5eaf-4df6-b5ca-303af8da9698 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:18.597429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:18.598326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:18.599186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 174/345] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:18.603797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846faa3-5eaf-4df6-b5ca-303af8da9698 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:18.616889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9347fa-d747-41e4-83ad-fcad18b3f707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:18.617864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9347fa-d747-41e4-83ad-fcad18b3f707 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:18.618340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9347fa-d747-41e4-83ad-fcad18b3f707 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:18.618805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9347fa-d747-41e4-83ad-fcad18b3f707 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:18.741466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9347fa-d747-41e4-83ad-fcad18b3f707 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:18.743643 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9347fa-d747-41e4-83ad-fcad18b3f707 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:18.768254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846faa3-5eaf-4df6-b5ca-303af8da9698 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:18.770318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:18.814940 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb64e1-7c9c-48ab-9bfb-6632f905f3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:18.815996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb64e1-7c9c-48ab-9bfb-6632f905f3ad None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:18.816472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb64e1-7c9c-48ab-9bfb-6632f905f3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:18.817030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb64e1-7c9c-48ab-9bfb-6632f905f3ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:18.830595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb64e1-7c9c-48ab-9bfb-6632f905f3ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-19fb64e1-7c9c-48ab-9bfb-6632f905f3ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:18.836246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:18.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:18.837951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:18.840574 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:18.841240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 175/346] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:48:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:18.842838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb64e1-7c9c-48ab-9bfb-6632f905f3ad None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:18.868027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 172/347] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:18.868800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9347fa-d747-41e4-83ad-fcad18b3f707 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:18.979734 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb64e1-7c9c-48ab-9bfb-6632f905f3ad None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:19.002869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9347fa-d747-41e4-83ad-fcad18b3f707 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:19.029237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50117273-97bf-424d-9b07-b0074d7c6077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:19.030751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50117273-97bf-424d-9b07-b0074d7c6077 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:19.031259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50117273-97bf-424d-9b07-b0074d7c6077 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:19.031732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50117273-97bf-424d-9b07-b0074d7c6077 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:19.032662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50117273-97bf-424d-9b07-b0074d7c6077 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:19.033258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50117273-97bf-424d-9b07-b0074d7c6077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:19.034839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50117273-97bf-424d-9b07-b0074d7c6077 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:19.045094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 176/348] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:19.047290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50117273-97bf-424d-9b07-b0074d7c6077 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:19.057002 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14e420-9cc1-49c1-9134-b649107e7d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:19.057655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e420-9cc1-49c1-9134-b649107e7d33 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:19.057934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e420-9cc1-49c1-9134-b649107e7d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:19.058207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e420-9cc1-49c1-9134-b649107e7d33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:19.075704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14e420-9cc1-49c1-9134-b649107e7d33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:19.081490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fc14e420-9cc1-49c1-9134-b649107e7d33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:19.083499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:19.084420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:19.085231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 173/349] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:19.090608 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14e420-9cc1-49c1-9134-b649107e7d33 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:19.147073 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:19.179339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50117273-97bf-424d-9b07-b0074d7c6077 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:19.223766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc59d4-0c40-4312-a7f6-ccc1c6b0fd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:19.224732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc59d4-0c40-4312-a7f6-ccc1c6b0fd89 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:19.225213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc59d4-0c40-4312-a7f6-ccc1c6b0fd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:19.225737 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc59d4-0c40-4312-a7f6-ccc1c6b0fd89 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:19.226406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14e420-9cc1-49c1-9134-b649107e7d33 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:19.226891 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc59d4-0c40-4312-a7f6-ccc1c6b0fd89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:19.227475 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc59d4-0c40-4312-a7f6-ccc1c6b0fd89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:19.229064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc59d4-0c40-4312-a7f6-ccc1c6b0fd89 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:19.240199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 177/350] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:19.241458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc59d4-0c40-4312-a7f6-ccc1c6b0fd89 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:19.270126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9717371-a706-46ba-91e3-1aedb6fd7181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:19.272594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9717371-a706-46ba-91e3-1aedb6fd7181 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:19.272881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9717371-a706-46ba-91e3-1aedb6fd7181 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:19.273157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9717371-a706-46ba-91e3-1aedb6fd7181 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:19.275573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9717371-a706-46ba-91e3-1aedb6fd7181 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:19.277030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9717371-a706-46ba-91e3-1aedb6fd7181 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:19.281312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 174/351] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:19 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:19.282313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9717371-a706-46ba-91e3-1aedb6fd7181 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:19.362007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:19.392025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc59d4-0c40-4312-a7f6-ccc1c6b0fd89 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:19.417486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9717371-a706-46ba-91e3-1aedb6fd7181 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:19.431202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:19.449021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dc8c2-5190-4941-b24c-e2b3f0651cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:19.449666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dc8c2-5190-4941-b24c-e2b3f0651cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:19.449949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dc8c2-5190-4941-b24c-e2b3f0651cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:19.450224 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dc8c2-5190-4941-b24c-e2b3f0651cf0 None admin] PATH_INFO: `/v3/users/7f993e0d69314fac8ed65a232e07e28d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:19.451718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dc8c2-5190-4941-b24c-e2b3f0651cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=7f993e0d69314fac8ed65a232e07e28d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:19.453181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dc8c2-5190-4941-b24c-e2b3f0651cf0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:19.456361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 178/352] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:48:19 2022] GET /identity/v3/users/7f993e0d69314fac8ed65a232e07e28d => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:19.457181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dc8c2-5190-4941-b24c-e2b3f0651cf0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:19.462386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad97de-7da9-47c4-876f-ba0780eebd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:19.463034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad97de-7da9-47c4-876f-ba0780eebd37 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:19.463310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad97de-7da9-47c4-876f-ba0780eebd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:19.463582 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad97de-7da9-47c4-876f-ba0780eebd37 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:19.464249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad97de-7da9-47c4-876f-ba0780eebd37 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:19.464548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad97de-7da9-47c4-876f-ba0780eebd37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:19.465932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad97de-7da9-47c4-876f-ba0780eebd37 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:19.477184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 175/353] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:19.478676 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad97de-7da9-47c4-876f-ba0780eebd37 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:19.591253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dc8c2-5190-4941-b24c-e2b3f0651cf0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:19.624716 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad97de-7da9-47c4-876f-ba0780eebd37 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:19.636437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09671894-d02a-4f01-8e92-5f27b3455b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:19.637052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09671894-d02a-4f01-8e92-5f27b3455b58 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:19.637331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09671894-d02a-4f01-8e92-5f27b3455b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:19.637602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09671894-d02a-4f01-8e92-5f27b3455b58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:19.672446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4b971-5028-403d-a22b-09eb9c4f70cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:19.673191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4b971-5028-403d-a22b-09eb9c4f70cd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:19.673477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4b971-5028-403d-a22b-09eb9c4f70cd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:19.673755 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4b971-5028-403d-a22b-09eb9c4f70cd None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:19.674141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:19.688987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4b971-5028-403d-a22b-09eb9c4f70cd None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:19.689840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4b971-5028-403d-a22b-09eb9c4f70cd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:19.692168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 176/354] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:19 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:19.693165 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da481411-cfc4-4c7f-b4f3-f55abff0d6fa None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:19.693452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da481411-cfc4-4c7f-b4f3-f55abff0d6fa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:19.693980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da481411-cfc4-4c7f-b4f3-f55abff0d6fa None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:19.695483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 177/355] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:19.697788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da481411-cfc4-4c7f-b4f3-f55abff0d6fa None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:19.738723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09671894-d02a-4f01-8e92-5f27b3455b58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:19.740559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09671894-d02a-4f01-8e92-5f27b3455b58 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:19.746823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:19.824197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da481411-cfc4-4c7f-b4f3-f55abff0d6fa None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:19.868697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c0469-4f89-4d9c-ba93-472b8e85fef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:19.869145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c0469-4f89-4d9c-ba93-472b8e85fef6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:19.869424 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c0469-4f89-4d9c-ba93-472b8e85fef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:19.869701 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c0469-4f89-4d9c-ba93-472b8e85fef6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:19.870426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c0469-4f89-4d9c-ba93-472b8e85fef6 None admin] RBAC: Adding query filter params (role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d, user.id=7f993e0d69314fac8ed65a232e07e28d, scope.system=all) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:19.870734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c0469-4f89-4d9c-ba93-472b8e85fef6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:19.872203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c0469-4f89-4d9c-ba93-472b8e85fef6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:19.874030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 179/356] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:19.875083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09671894-d02a-4f01-8e92-5f27b3455b58 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:19.885087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 178/357] 10.222.0.86 () {56 vars in 1308 bytes} [Mon Aug 29 18:48:19 2022] GET /identity/v3/role_assignments?user.id=7f993e0d69314fac8ed65a232e07e28d&scope.system=all&role.id=4b69aa55fbbf4624b0b6036a5a8b2d4d => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:19.886256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c0469-4f89-4d9c-ba93-472b8e85fef6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:19.960953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:20.022813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:20.032025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09671894-d02a-4f01-8e92-5f27b3455b58 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:20.034436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c0469-4f89-4d9c-ba93-472b8e85fef6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:20.078473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46892bc6-7cd6-49ed-a1b5-67659156906e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:20.079152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46892bc6-7cd6-49ed-a1b5-67659156906e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.079408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46892bc6-7cd6-49ed-a1b5-67659156906e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.079685 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46892bc6-7cd6-49ed-a1b5-67659156906e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:20.082105 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2d32c-79ea-4e28-be85-d86d944935b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:20.082725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2d32c-79ea-4e28-be85-d86d944935b8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.083046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2d32c-79ea-4e28-be85-d86d944935b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.083304 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2d32c-79ea-4e28-be85-d86d944935b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:20.084040 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2d32c-79ea-4e28-be85-d86d944935b8 None admin] RBAC: Adding query filter params (group.id=1865f727726848c8bad00c83f6f673eb, role.id=a0db5bd603fc4096917bbcd8c6932bc0, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:20.084355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2d32c-79ea-4e28-be85-d86d944935b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:20.085790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2d32c-79ea-4e28-be85-d86d944935b8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:20.092553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 179/358] 10.222.0.86 () {56 vars in 1376 bytes} [Mon Aug 29 18:48:19 2022] GET /identity/v3/role_assignments?group.id=1865f727726848c8bad00c83f6f673eb&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:20.099071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5d6d2-189c-4194-af41-36454337e51d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.099071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5d6d2-189c-4194-af41-36454337e51d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.099071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5d6d2-189c-4194-af41-36454337e51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:20.118063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-11d5d6d2-189c-4194-af41-36454337e51d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:20.119546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11d5d6d2-189c-4194-af41-36454337e51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:20.183387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46892bc6-7cd6-49ed-a1b5-67659156906e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:20.184783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46892bc6-7cd6-49ed-a1b5-67659156906e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:20.218419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 180/359] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:20.219588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98317831-6dff-4613-b99c-218ac4e791a5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.219866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98317831-6dff-4613-b99c-218ac4e791a5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.220141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98317831-6dff-4613-b99c-218ac4e791a5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:20.221440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 181/360] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:20.222125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98317831-6dff-4613-b99c-218ac4e791a5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:20.256155 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:20.305337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 180/361] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:20.306557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2446980-5b16-4962-8acf-551638484f57 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.306903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2446980-5b16-4962-8acf-551638484f57 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.307215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2446980-5b16-4962-8acf-551638484f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:20.325996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c2446980-5b16-4962-8acf-551638484f57 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:20.328887 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c2446980-5b16-4962-8acf-551638484f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:20.330526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:20.350293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98317831-6dff-4613-b99c-218ac4e791a5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:20.388339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b19f0-c4a0-442a-9702-82a97dcaf710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:20.390019 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b19f0-c4a0-442a-9702-82a97dcaf710 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.390347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b19f0-c4a0-442a-9702-82a97dcaf710 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.390618 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b19f0-c4a0-442a-9702-82a97dcaf710 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:20.391904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b19f0-c4a0-442a-9702-82a97dcaf710 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:20.393470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b19f0-c4a0-442a-9702-82a97dcaf710 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:20.395139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 182/362] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:20 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:20.395951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b19f0-c4a0-442a-9702-82a97dcaf710 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:20.402174 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 181/363] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:20.403048 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2446980-5b16-4962-8acf-551638484f57 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:20.505077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:20.524300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2446980-5b16-4962-8acf-551638484f57 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:20.570242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c34fc3-7cca-41ef-b520-c000b4712790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:20.571561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c34fc3-7cca-41ef-b520-c000b4712790 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.572204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c34fc3-7cca-41ef-b520-c000b4712790 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.572996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c34fc3-7cca-41ef-b520-c000b4712790 None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:20.574713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c34fc3-7cca-41ef-b520-c000b4712790 None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:20.576426 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c34fc3-7cca-41ef-b520-c000b4712790 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:20.577003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b19f0-c4a0-442a-9702-82a97dcaf710 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:20.578926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 182/364] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:20 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:20.580894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c34fc3-7cca-41ef-b520-c000b4712790 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:20.614258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ee353-0e0a-4f14-9f0c-3b8edd977912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:20.614922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee353-0e0a-4f14-9f0c-3b8edd977912 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.615241 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee353-0e0a-4f14-9f0c-3b8edd977912 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.615514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee353-0e0a-4f14-9f0c-3b8edd977912 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:20.616651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ee353-0e0a-4f14-9f0c-3b8edd977912 None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:20.618091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ee353-0e0a-4f14-9f0c-3b8edd977912 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:20.619838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 183/365] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:20 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:20.620723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ee353-0e0a-4f14-9f0c-3b8edd977912 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:20.655417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:20.726322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c34fc3-7cca-41ef-b520-c000b4712790 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:20.745190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ee353-0e0a-4f14-9f0c-3b8edd977912 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:20.776176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78575896-3010-4567-b877-dda9ed8c1c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:20.777205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78575896-3010-4567-b877-dda9ed8c1c6f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.777680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78575896-3010-4567-b877-dda9ed8c1c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.778170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78575896-3010-4567-b877-dda9ed8c1c6f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:20.792653 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78575896-3010-4567-b877-dda9ed8c1c6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:20.796880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b7d61-5640-4bf2-bb46-bfbd88029d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:20.796880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b7d61-5640-4bf2-bb46-bfbd88029d9a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.796880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b7d61-5640-4bf2-bb46-bfbd88029d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.796880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b7d61-5640-4bf2-bb46-bfbd88029d9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:20.797197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b7d61-5640-4bf2-bb46-bfbd88029d9a None admin] RBAC: Adding query filter params (group.id=1865f727726848c8bad00c83f6f673eb, role.id=31cc9f3a35594d40924277f4778c9f03, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:20.797197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b7d61-5640-4bf2-bb46-bfbd88029d9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:20.797715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b7d61-5640-4bf2-bb46-bfbd88029d9a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-78575896-3010-4567-b877-dda9ed8c1c6f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:20.801981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:20.803149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:20.804013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 183/366] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:48:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:20.806921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 184/367] 10.222.0.86 () {56 vars in 1376 bytes} [Mon Aug 29 18:48:20 2022] GET /identity/v3/role_assignments?group.id=1865f727726848c8bad00c83f6f673eb&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=31cc9f3a35594d40924277f4778c9f03 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:20.807021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78575896-3010-4567-b877-dda9ed8c1c6f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:20.807106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b7d61-5640-4bf2-bb46-bfbd88029d9a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:20.825666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:20.919900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:20.926312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78575896-3010-4567-b877-dda9ed8c1c6f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:20.937696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b7d61-5640-4bf2-bb46-bfbd88029d9a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:20.968796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e7bc3-8f6b-474a-829c-92fed0aca522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:20.969463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e7bc3-8f6b-474a-829c-92fed0aca522 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.969750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e7bc3-8f6b-474a-829c-92fed0aca522 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.970028 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e7bc3-8f6b-474a-829c-92fed0aca522 None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:20.971446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e7bc3-8f6b-474a-829c-92fed0aca522 None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:20.973639 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e7bc3-8f6b-474a-829c-92fed0aca522 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:20.977717 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 184/368] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:20 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:20.977717 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e7bc3-8f6b-474a-829c-92fed0aca522 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:20.994800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839ec6f-5625-418a-95ec-642211cded32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:20.995613 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839ec6f-5625-418a-95ec-642211cded32 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:20.995961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839ec6f-5625-418a-95ec-642211cded32 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:20.996303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839ec6f-5625-418a-95ec-642211cded32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:21.034176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839ec6f-5625-418a-95ec-642211cded32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9839ec6f-5625-418a-95ec-642211cded32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:21.041853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:21.042929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:21.043863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:21.044612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 185/369] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:48:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:21.044898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839ec6f-5625-418a-95ec-642211cded32 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:21.085849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:21.106841 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e7bc3-8f6b-474a-829c-92fed0aca522 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:21.144400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ef881-a1f4-4f3b-9ab1-5bdd12135d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:21.145351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ef881-a1f4-4f3b-9ab1-5bdd12135d9f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:21.145830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ef881-a1f4-4f3b-9ab1-5bdd12135d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:21.146245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ef881-a1f4-4f3b-9ab1-5bdd12135d9f None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:21.147079 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ef881-a1f4-4f3b-9ab1-5bdd12135d9f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:21.147568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ef881-a1f4-4f3b-9ab1-5bdd12135d9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:21.149282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ef881-a1f4-4f3b-9ab1-5bdd12135d9f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:21.159471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 185/370] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:21.160681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ef881-a1f4-4f3b-9ab1-5bdd12135d9f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:21.168606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839ec6f-5625-418a-95ec-642211cded32 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:21.207845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b689e27-1312-43fc-b088-7fd6a3c4bcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:21.208465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b689e27-1312-43fc-b088-7fd6a3c4bcaf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:21.208747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b689e27-1312-43fc-b088-7fd6a3c4bcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:21.209030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b689e27-1312-43fc-b088-7fd6a3c4bcaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:21.209758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b689e27-1312-43fc-b088-7fd6a3c4bcaf None admin] RBAC: Adding query filter params (group.id=1865f727726848c8bad00c83f6f673eb, role.id=a0db5bd603fc4096917bbcd8c6932bc0, scope.project.id=f456bff4d42f4ff194d2ac607af555db) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:21.210133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b689e27-1312-43fc-b088-7fd6a3c4bcaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:21.211708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b689e27-1312-43fc-b088-7fd6a3c4bcaf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:21.218321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 186/371] 10.222.0.86 () {56 vars in 1376 bytes} [Mon Aug 29 18:48:21 2022] GET /identity/v3/role_assignments?group.id=1865f727726848c8bad00c83f6f673eb&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:21.218989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b689e27-1312-43fc-b088-7fd6a3c4bcaf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:21.246964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:21.285035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ef881-a1f4-4f3b-9ab1-5bdd12135d9f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:21.327212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d96b71-90b3-4127-8176-30fea94605f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:21.328226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d96b71-90b3-4127-8176-30fea94605f5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:21.328683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d96b71-90b3-4127-8176-30fea94605f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:21.329127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d96b71-90b3-4127-8176-30fea94605f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:21.330075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d96b71-90b3-4127-8176-30fea94605f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:21.330555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d96b71-90b3-4127-8176-30fea94605f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:21.332210 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d96b71-90b3-4127-8176-30fea94605f5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:21.343298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 186/372] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:21.349232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d96b71-90b3-4127-8176-30fea94605f5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:21.354304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:21.405663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b689e27-1312-43fc-b088-7fd6a3c4bcaf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:21.475609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d96b71-90b3-4127-8176-30fea94605f5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:21.516812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1cf70-4a4d-4e31-a9c6-569052a3b24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:21.517992 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1cf70-4a4d-4e31-a9c6-569052a3b24d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:21.518451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1cf70-4a4d-4e31-a9c6-569052a3b24d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:21.518901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1cf70-4a4d-4e31-a9c6-569052a3b24d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:21.520274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9055ce-2caa-4e7f-9fc6-27b0b58f08ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:21.520901 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9055ce-2caa-4e7f-9fc6-27b0b58f08ab None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:21.521184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9055ce-2caa-4e7f-9fc6-27b0b58f08ab None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:21.521461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9055ce-2caa-4e7f-9fc6-27b0b58f08ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:21.609434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1cf70-4a4d-4e31-a9c6-569052a3b24d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:21.611063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9055ce-2caa-4e7f-9fc6-27b0b58f08ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:21.611765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1cf70-4a4d-4e31-a9c6-569052a3b24d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:21.612578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9055ce-2caa-4e7f-9fc6-27b0b58f08ab None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:21.615172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:21.637420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:21.748984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 187/373] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:21.755335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9055ce-2caa-4e7f-9fc6-27b0b58f08ab None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:21.755807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 187/374] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:21.761288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1cf70-4a4d-4e31-a9c6-569052a3b24d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:21.888191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9055ce-2caa-4e7f-9fc6-27b0b58f08ab None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:21.895222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1cf70-4a4d-4e31-a9c6-569052a3b24d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:21.906181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:21.910080 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:21.952034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9787ca0-cf7d-4611-ae98-ef33153d5195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:21.952685 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9787ca0-cf7d-4611-ae98-ef33153d5195 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:21.952962 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9787ca0-cf7d-4611-ae98-ef33153d5195 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:21.953233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9787ca0-cf7d-4611-ae98-ef33153d5195 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:21.954640 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9787ca0-cf7d-4611-ae98-ef33153d5195 None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:21.956133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9787ca0-cf7d-4611-ae98-ef33153d5195 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:21.958795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 188/375] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:21 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:21.962149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f8cb2-3a45-438f-a12e-b3d77e098285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:21.962794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f8cb2-3a45-438f-a12e-b3d77e098285 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:21.963106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f8cb2-3a45-438f-a12e-b3d77e098285 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:21.963385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f8cb2-3a45-438f-a12e-b3d77e098285 None admin] PATH_INFO: `/v3/projects/3e9f996197414d24b783bd100f720103` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:21.964696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9787ca0-cf7d-4611-ae98-ef33153d5195 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:21.966064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f8cb2-3a45-438f-a12e-b3d77e098285 None admin] RBAC: Authorizing `identity:get_project(project_id=3e9f996197414d24b783bd100f720103)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:21.967567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f8cb2-3a45-438f-a12e-b3d77e098285 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:21.969227 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 188/376] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:21 2022] GET /identity/v3/projects/3e9f996197414d24b783bd100f720103 => generated 344 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:21.978246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f8cb2-3a45-438f-a12e-b3d77e098285 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:22.145822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9787ca0-cf7d-4611-ae98-ef33153d5195 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:22.161010 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f8cb2-3a45-438f-a12e-b3d77e098285 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:22.200573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c489c6b-5d53-4f61-bcac-c494b5950a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:22.201538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c489c6b-5d53-4f61-bcac-c494b5950a2f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:22.201817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c489c6b-5d53-4f61-bcac-c494b5950a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:22.202099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c489c6b-5d53-4f61-bcac-c494b5950a2f None admin] PATH_INFO: `/v3/groups/3b936a05b35d45109e811fc637afa7df` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:22.208825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c489c6b-5d53-4f61-bcac-c494b5950a2f None admin] RBAC: Authorizing `identity:get_group(group_id=3b936a05b35d45109e811fc637afa7df)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:22.210471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c489c6b-5d53-4f61-bcac-c494b5950a2f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:22.212325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 189/377] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:21 2022] GET /identity/v3/groups/3b936a05b35d45109e811fc637afa7df => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:22.220477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c489c6b-5d53-4f61-bcac-c494b5950a2f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:22.231630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7b2fb-32ca-4834-9c89-dc889332b23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:22.232493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7b2fb-32ca-4834-9c89-dc889332b23b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:22.233328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7b2fb-32ca-4834-9c89-dc889332b23b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:22.233605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7b2fb-32ca-4834-9c89-dc889332b23b None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:22.238209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7b2fb-32ca-4834-9c89-dc889332b23b None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:22.239775 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7b2fb-32ca-4834-9c89-dc889332b23b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:22.241609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 189/378] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:21 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:22.249215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7b2fb-32ca-4834-9c89-dc889332b23b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:22.317915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:22.346237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:22.380679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c489c6b-5d53-4f61-bcac-c494b5950a2f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:22.441732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7b2fb-32ca-4834-9c89-dc889332b23b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:22.456475 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc341a7f-1991-45f1-81c9-e48204716cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:22.457403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc341a7f-1991-45f1-81c9-e48204716cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:22.457958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc341a7f-1991-45f1-81c9-e48204716cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:22.458411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc341a7f-1991-45f1-81c9-e48204716cd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:22.459357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc341a7f-1991-45f1-81c9-e48204716cd1 None admin] RBAC: Adding query filter params (group.id=3b936a05b35d45109e811fc637afa7df, role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=3e9f996197414d24b783bd100f720103) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:22.459848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc341a7f-1991-45f1-81c9-e48204716cd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:22.461590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc341a7f-1991-45f1-81c9-e48204716cd1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:22.469049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 190/379] 10.222.0.86 () {56 vars in 1376 bytes} [Mon Aug 29 18:48:22 2022] GET /identity/v3/role_assignments?group.id=3b936a05b35d45109e811fc637afa7df&scope.project.id=3e9f996197414d24b783bd100f720103&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:22.470588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32000bbb-fef4-4e70-8224-45483ef4081f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:22.471134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32000bbb-fef4-4e70-8224-45483ef4081f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:22.471678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32000bbb-fef4-4e70-8224-45483ef4081f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:22.473298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 191/380] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:22.479435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25310e5f-02ee-4241-bf30-d7c3b15803f8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:22.480001 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25310e5f-02ee-4241-bf30-d7c3b15803f8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:22.480453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25310e5f-02ee-4241-bf30-d7c3b15803f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:22.493334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9693c-52b3-407a-9525-f39f7d8eafc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:22.493970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9693c-52b3-407a-9525-f39f7d8eafc3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:22.494246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9693c-52b3-407a-9525-f39f7d8eafc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:22.494516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9693c-52b3-407a-9525-f39f7d8eafc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:22.495289 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9693c-52b3-407a-9525-f39f7d8eafc3 None admin] RBAC: Adding query filter params (group.id=1865f727726848c8bad00c83f6f673eb, role.id=31cc9f3a35594d40924277f4778c9f03, scope.project.id=f456bff4d42f4ff194d2ac607af555db) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:22.495598 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9693c-52b3-407a-9525-f39f7d8eafc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:22.498814 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9693c-52b3-407a-9525-f39f7d8eafc3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:22.505795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 190/381] 10.222.0.86 () {56 vars in 1376 bytes} [Mon Aug 29 18:48:22 2022] GET /identity/v3/role_assignments?group.id=1865f727726848c8bad00c83f6f673eb&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=31cc9f3a35594d40924277f4778c9f03 => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:22.518970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-25310e5f-02ee-4241-bf30-d7c3b15803f8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:22.520963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-25310e5f-02ee-4241-bf30-d7c3b15803f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:22.629468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 192/382] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:22.953442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9693c-52b3-407a-9525-f39f7d8eafc3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:23.079175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9693c-52b3-407a-9525-f39f7d8eafc3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:23.093114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:23.127727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cb734-ad55-47ce-9bd1-8526cf0d0bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:23.128377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cb734-ad55-47ce-9bd1-8526cf0d0bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:23.128661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cb734-ad55-47ce-9bd1-8526cf0d0bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:23.128936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cb734-ad55-47ce-9bd1-8526cf0d0bf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:23.142213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cb734-ad55-47ce-9bd1-8526cf0d0bf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ed4cb734-ad55-47ce-9bd1-8526cf0d0bf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:23.147624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:23.148495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:23.149664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 191/383] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:23.155471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25310e5f-02ee-4241-bf30-d7c3b15803f8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:23.207980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:23.222603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d60d0-5f0f-47ab-b698-5d28e8924a23 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:23.222887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d60d0-5f0f-47ab-b698-5d28e8924a23 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:23.223194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d60d0-5f0f-47ab-b698-5d28e8924a23 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:23.224786 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 192/384] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:23.243674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30df54d-c6e3-45df-a2b9-107058a22057 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:23.243969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30df54d-c6e3-45df-a2b9-107058a22057 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:23.244223 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30df54d-c6e3-45df-a2b9-107058a22057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:23.269093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f30df54d-c6e3-45df-a2b9-107058a22057 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:23.270297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f30df54d-c6e3-45df-a2b9-107058a22057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:23.330179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25310e5f-02ee-4241-bf30-d7c3b15803f8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:23.392916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 193/385] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:23.412665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9f3c5-fa2f-4bbc-b545-1effffce3743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:23.413367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9f3c5-fa2f-4bbc-b545-1effffce3743 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:23.413655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9f3c5-fa2f-4bbc-b545-1effffce3743 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:23.413975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9f3c5-fa2f-4bbc-b545-1effffce3743 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:23.414652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9f3c5-fa2f-4bbc-b545-1effffce3743 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:23.414956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9f3c5-fa2f-4bbc-b545-1effffce3743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:23.416359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9f3c5-fa2f-4bbc-b545-1effffce3743 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:23.431116 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 193/386] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:23.436784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30df54d-c6e3-45df-a2b9-107058a22057 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:23.544948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:23.548742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30df54d-c6e3-45df-a2b9-107058a22057 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:23.598222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02975757-5dd7-4532-887e-35f35d8082ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:23.598869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02975757-5dd7-4532-887e-35f35d8082ec None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:23.599175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02975757-5dd7-4532-887e-35f35d8082ec None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:23.599453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02975757-5dd7-4532-887e-35f35d8082ec None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:23.600598 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02975757-5dd7-4532-887e-35f35d8082ec None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:23.602221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02975757-5dd7-4532-887e-35f35d8082ec None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:23.604135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 194/387] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:23 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:23.609800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9f3c5-fa2f-4bbc-b545-1effffce3743 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:23.736726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:23.751391 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9f3c5-fa2f-4bbc-b545-1effffce3743 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:23.756669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02975757-5dd7-4532-887e-35f35d8082ec None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:23.833568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2c585-f185-4186-af5a-7b64a3dca0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:23.834137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2c585-f185-4186-af5a-7b64a3dca0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:23.834418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2c585-f185-4186-af5a-7b64a3dca0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:23.834692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2c585-f185-4186-af5a-7b64a3dca0b1 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:23.835951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2c585-f185-4186-af5a-7b64a3dca0b1 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:23.837409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2c585-f185-4186-af5a-7b64a3dca0b1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:23.839059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 194/388] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:23 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:23.860994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2c585-f185-4186-af5a-7b64a3dca0b1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:23.912001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02975757-5dd7-4532-887e-35f35d8082ec None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:23.952462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccffc4b-4f28-41ee-8bb1-2941b8ce1195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:23.953097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccffc4b-4f28-41ee-8bb1-2941b8ce1195 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:23.953373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccffc4b-4f28-41ee-8bb1-2941b8ce1195 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:23.953645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccffc4b-4f28-41ee-8bb1-2941b8ce1195 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:23.977496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccffc4b-4f28-41ee-8bb1-2941b8ce1195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0ccffc4b-4f28-41ee-8bb1-2941b8ce1195 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:23.984469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:23.985426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:23.986662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 195/389] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:48:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:23.997426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccffc4b-4f28-41ee-8bb1-2941b8ce1195 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:24.042242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:24.090716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2c585-f185-4186-af5a-7b64a3dca0b1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:24.143363 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873e324-55df-44f4-b274-953cd0e8e71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:24.144031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873e324-55df-44f4-b274-953cd0e8e71d None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:24.144315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873e324-55df-44f4-b274-953cd0e8e71d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:24.144594 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873e324-55df-44f4-b274-953cd0e8e71d None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/groups/1865f727726848c8bad00c83f6f673eb/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:24.146689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873e324-55df-44f4-b274-953cd0e8e71d None admin] RBAC: Authorizing `identity:create_grant(project_id=cdbff3dc80ab4433ba8d78b52eb66086, group_id=1865f727726848c8bad00c83f6f673eb, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:24.148287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873e324-55df-44f4-b274-953cd0e8e71d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:24.159284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccffc4b-4f28-41ee-8bb1-2941b8ce1195 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:24.166880 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 195/390] 10.222.0.86 () {58 vars in 1442 bytes} [Mon Aug 29 18:48:23 2022] PUT /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/groups/1865f727726848c8bad00c83f6f673eb/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:48:24.171015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef745f89-2f70-437a-b05a-fe8b09421be9 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:24.171015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef745f89-2f70-437a-b05a-fe8b09421be9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:24.171015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef745f89-2f70-437a-b05a-fe8b09421be9 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:24.171015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 196/391] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:24.180468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef96e0-43f4-493e-a5f4-11d30aace0f8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:24.180878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef96e0-43f4-493e-a5f4-11d30aace0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:24.181233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef96e0-43f4-493e-a5f4-11d30aace0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:24.209916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e9ef96e0-43f4-493e-a5f4-11d30aace0f8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:24.211741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e9ef96e0-43f4-493e-a5f4-11d30aace0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:24.241033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c15fbf-ae17-416c-a1bf-6c0244e7c0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:24.244402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c15fbf-ae17-416c-a1bf-6c0244e7c0ca None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:24.244816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c15fbf-ae17-416c-a1bf-6c0244e7c0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:24.245098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c15fbf-ae17-416c-a1bf-6c0244e7c0ca None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:24.247021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c15fbf-ae17-416c-a1bf-6c0244e7c0ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:24.248741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c15fbf-ae17-416c-a1bf-6c0244e7c0ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:24.249811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:24.254051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c15fbf-ae17-416c-a1bf-6c0244e7c0ca None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:24.269811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 196/392] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:24.275563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c15fbf-ae17-416c-a1bf-6c0244e7c0ca None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:24.384335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 197/393] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:24.394721 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:24.409678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c15fbf-ae17-416c-a1bf-6c0244e7c0ca None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:24.447071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfab29-d873-4a1f-8037-d8c1d574c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:24.447801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfab29-d873-4a1f-8037-d8c1d574c792 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:24.448086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfab29-d873-4a1f-8037-d8c1d574c792 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:24.448362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfab29-d873-4a1f-8037-d8c1d574c792 None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:24.449575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfab29-d873-4a1f-8037-d8c1d574c792 None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:24.451111 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfab29-d873-4a1f-8037-d8c1d574c792 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:24.453010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 197/394] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:24 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:24.459953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef96e0-43f4-493e-a5f4-11d30aace0f8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:24.573245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef96e0-43f4-493e-a5f4-11d30aace0f8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:24.622478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01eae4b-a62a-4194-a4e1-45069ce2abbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:24.623180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01eae4b-a62a-4194-a4e1-45069ce2abbc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:24.623512 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01eae4b-a62a-4194-a4e1-45069ce2abbc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:24.623789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01eae4b-a62a-4194-a4e1-45069ce2abbc None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:24.624930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01eae4b-a62a-4194-a4e1-45069ce2abbc None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:24.626427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01eae4b-a62a-4194-a4e1-45069ce2abbc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:24.628268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 198/395] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:24 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:24.636513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfab29-d873-4a1f-8037-d8c1d574c792 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:24.691018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01eae4b-a62a-4194-a4e1-45069ce2abbc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:24.713176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:24.838871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfab29-d873-4a1f-8037-d8c1d574c792 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:24.843836 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01eae4b-a62a-4194-a4e1-45069ce2abbc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:24.897128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17ec05-62d1-4025-84f0-873de03cbc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:24.897719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17ec05-62d1-4025-84f0-873de03cbc92 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:24.898002 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17ec05-62d1-4025-84f0-873de03cbc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:24.898277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17ec05-62d1-4025-84f0-873de03cbc92 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/groups/1865f727726848c8bad00c83f6f673eb/roles/31cc9f3a35594d40924277f4778c9f03` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:24.906055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17ec05-62d1-4025-84f0-873de03cbc92 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdbff3dc80ab4433ba8d78b52eb66086, group_id=1865f727726848c8bad00c83f6f673eb, role_id=31cc9f3a35594d40924277f4778c9f03)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:24.907810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17ec05-62d1-4025-84f0-873de03cbc92 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:24.919480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d037d-de87-4eca-a308-33849c33c5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:24.920229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d037d-de87-4eca-a308-33849c33c5bd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:24.920541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d037d-de87-4eca-a308-33849c33c5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:24.920817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d037d-de87-4eca-a308-33849c33c5bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:24.933745 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 198/396] 10.222.0.86 () {58 vars in 1442 bytes} [Mon Aug 29 18:48:24 2022] PUT /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086/groups/1865f727726848c8bad00c83f6f673eb/roles/31cc9f3a35594d40924277f4778c9f03 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:48:24.939804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d037d-de87-4eca-a308-33849c33c5bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b73d037d-de87-4eca-a308-33849c33c5bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:24.945330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:24.946371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:24.948047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 199/397] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:24.954496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17ec05-62d1-4025-84f0-873de03cbc92 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:24.984067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:25.103579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17ec05-62d1-4025-84f0-873de03cbc92 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:25.119215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a7c05-64d0-4548-a7de-a6091debf232 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:25.119547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a7c05-64d0-4548-a7de-a6091debf232 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:25.119761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a7c05-64d0-4548-a7de-a6091debf232 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:25.121359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 200/398] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:48:25.129027 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850a34-9834-4f36-97a7-15dbc05b3243 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:25.129324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850a34-9834-4f36-97a7-15dbc05b3243 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:25.129576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850a34-9834-4f36-97a7-15dbc05b3243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:25.158456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fb850a34-9834-4f36-97a7-15dbc05b3243 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:25.159629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb850a34-9834-4f36-97a7-15dbc05b3243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:25.180260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1698b66-01eb-429d-98b9-a8555ef73f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:25.181173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698b66-01eb-429d-98b9-a8555ef73f49 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:25.181592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698b66-01eb-429d-98b9-a8555ef73f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:25.182002 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698b66-01eb-429d-98b9-a8555ef73f49 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:25.183233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1698b66-01eb-429d-98b9-a8555ef73f49 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:25.183708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1698b66-01eb-429d-98b9-a8555ef73f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:25.186071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1698b66-01eb-429d-98b9-a8555ef73f49 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:25.200861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 199/399] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:25.210300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79c9cd-87bf-4501-a6b8-3a22147d09b2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:25.210300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79c9cd-87bf-4501-a6b8-3a22147d09b2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:25.210300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79c9cd-87bf-4501-a6b8-3a22147d09b2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:25.210300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 200/400] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:25.211803 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79c9cd-87bf-4501-a6b8-3a22147d09b2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:25.240652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 201/401] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:25.243256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846bd93-f911-48c2-b9d1-aec05304438f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:25.243549 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846bd93-f911-48c2-b9d1-aec05304438f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:25.243827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846bd93-f911-48c2-b9d1-aec05304438f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:25.269242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0846bd93-f911-48c2-b9d1-aec05304438f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:25.270961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0846bd93-f911-48c2-b9d1-aec05304438f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:25.273679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:25.355171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:25.367272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 202/402] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:25.407661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79c9cd-87bf-4501-a6b8-3a22147d09b2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:25.460224 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b5b48-d79d-4961-96ed-28b76720a0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:25.461148 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b5b48-d79d-4961-96ed-28b76720a0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:25.461613 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b5b48-d79d-4961-96ed-28b76720a0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:25.462066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b5b48-d79d-4961-96ed-28b76720a0a6 None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:25.467914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b5b48-d79d-4961-96ed-28b76720a0a6 None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:25.469580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b5b48-d79d-4961-96ed-28b76720a0a6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:25.471389 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 201/403] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:25 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:25.477003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846bd93-f911-48c2-b9d1-aec05304438f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:25.536495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b5b48-d79d-4961-96ed-28b76720a0a6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:25.605055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846bd93-f911-48c2-b9d1-aec05304438f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:25.643998 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068736d-dd2e-48ef-80df-ec42f4bbe94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:25.644732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068736d-dd2e-48ef-80df-ec42f4bbe94b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:25.645085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068736d-dd2e-48ef-80df-ec42f4bbe94b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:25.645481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068736d-dd2e-48ef-80df-ec42f4bbe94b None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:25.647489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068736d-dd2e-48ef-80df-ec42f4bbe94b None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:25.649165 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068736d-dd2e-48ef-80df-ec42f4bbe94b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:25.651320 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 203/404] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:25 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:25.657087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068736d-dd2e-48ef-80df-ec42f4bbe94b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:25.675378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:25.738372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b5b48-d79d-4961-96ed-28b76720a0a6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:25.773604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64309366-38eb-49bf-a6fe-0ce940e5397e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:25.774253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64309366-38eb-49bf-a6fe-0ce940e5397e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:25.774534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64309366-38eb-49bf-a6fe-0ce940e5397e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:25.774809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64309366-38eb-49bf-a6fe-0ce940e5397e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:25.786867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64309366-38eb-49bf-a6fe-0ce940e5397e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:25.790482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068736d-dd2e-48ef-80df-ec42f4bbe94b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-64309366-38eb-49bf-a6fe-0ce940e5397e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:25.792242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:25.793084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 202/405] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:48:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:25.794253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64309366-38eb-49bf-a6fe-0ce940e5397e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:25.805200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:25.834244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef345f-676e-423e-a341-3ee8b5698549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:25.834908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef345f-676e-423e-a341-3ee8b5698549 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:25.835231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef345f-676e-423e-a341-3ee8b5698549 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:25.835502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef345f-676e-423e-a341-3ee8b5698549 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db/groups/1865f727726848c8bad00c83f6f673eb/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:25.837440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef345f-676e-423e-a341-3ee8b5698549 None admin] RBAC: Authorizing `identity:create_grant(project_id=f456bff4d42f4ff194d2ac607af555db, group_id=1865f727726848c8bad00c83f6f673eb, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:25.838962 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef345f-676e-423e-a341-3ee8b5698549 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:25.863196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 204/406] 10.222.0.86 () {58 vars in 1442 bytes} [Mon Aug 29 18:48:25 2022] PUT /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db/groups/1865f727726848c8bad00c83f6f673eb/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:48:25.864511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef345f-676e-423e-a341-3ee8b5698549 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:25.906975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64309366-38eb-49bf-a6fe-0ce940e5397e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:25.943910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10167d-4375-49c7-adec-d07c509e95b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:25.944551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10167d-4375-49c7-adec-d07c509e95b6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:25.944834 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10167d-4375-49c7-adec-d07c509e95b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:25.945111 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10167d-4375-49c7-adec-d07c509e95b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:25.957357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10167d-4375-49c7-adec-d07c509e95b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7b10167d-4375-49c7-adec-d07c509e95b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:25.963178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:25.965174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:25.966620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 203/407] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:25.971530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10167d-4375-49c7-adec-d07c509e95b6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:25.985446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:26.033877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef345f-676e-423e-a341-3ee8b5698549 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:26.061407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:26.092708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e1afd-495f-4056-9d5c-55aa1a5ca7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:26.093338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1afd-495f-4056-9d5c-55aa1a5ca7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:26.093619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1afd-495f-4056-9d5c-55aa1a5ca7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:26.093894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1afd-495f-4056-9d5c-55aa1a5ca7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:26.095114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e1afd-495f-4056-9d5c-55aa1a5ca7a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:26.095424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e1afd-495f-4056-9d5c-55aa1a5ca7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:26.097289 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e1afd-495f-4056-9d5c-55aa1a5ca7a8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:26.107534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10167d-4375-49c7-adec-d07c509e95b6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:26.108111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 205/408] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:26.113870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e1afd-495f-4056-9d5c-55aa1a5ca7a8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:26.145635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f25315-366f-41cb-95d1-3171598e973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:26.146635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f25315-366f-41cb-95d1-3171598e973e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:26.147079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f25315-366f-41cb-95d1-3171598e973e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:26.147554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f25315-366f-41cb-95d1-3171598e973e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:26.148396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f25315-366f-41cb-95d1-3171598e973e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:26.148941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f25315-366f-41cb-95d1-3171598e973e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:26.150511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f25315-366f-41cb-95d1-3171598e973e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:26.161480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 204/409] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:26.166736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f25315-366f-41cb-95d1-3171598e973e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:26.234667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e1afd-495f-4056-9d5c-55aa1a5ca7a8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:26.262557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:26.271432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c687a-66d3-4c88-b64b-746bb2f451bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:26.272092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c687a-66d3-4c88-b64b-746bb2f451bd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:26.272364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c687a-66d3-4c88-b64b-746bb2f451bd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:26.272720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c687a-66d3-4c88-b64b-746bb2f451bd None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:26.273906 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c687a-66d3-4c88-b64b-746bb2f451bd None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:26.275361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c687a-66d3-4c88-b64b-746bb2f451bd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:26.277242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 206/410] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:26 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:26.285083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c687a-66d3-4c88-b64b-746bb2f451bd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:26.287649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f25315-366f-41cb-95d1-3171598e973e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:26.326837 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:26.342494 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4723dc3-bd97-4876-9990-f402ccef4d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:26.343460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4723dc3-bd97-4876-9990-f402ccef4d55 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:26.343875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4723dc3-bd97-4876-9990-f402ccef4d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:26.344330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4723dc3-bd97-4876-9990-f402ccef4d55 None admin] PATH_INFO: `/v3/groups/3b936a05b35d45109e811fc637afa7df` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:26.346199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4723dc3-bd97-4876-9990-f402ccef4d55 None admin] RBAC: Authorizing `identity:get_group(group_id=3b936a05b35d45109e811fc637afa7df)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:26.349919 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4723dc3-bd97-4876-9990-f402ccef4d55 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:26.353041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 205/411] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:26 2022] GET /identity/v3/groups/3b936a05b35d45109e811fc637afa7df => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:26.359463 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4723dc3-bd97-4876-9990-f402ccef4d55 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:26.471008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4723dc3-bd97-4876-9990-f402ccef4d55 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:26.508644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7374d5-b32b-4f7f-88e7-90840d3d27b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:26.509542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7374d5-b32b-4f7f-88e7-90840d3d27b6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:26.509979 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7374d5-b32b-4f7f-88e7-90840d3d27b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:26.510392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7374d5-b32b-4f7f-88e7-90840d3d27b6 None admin] PATH_INFO: `/v3/projects/3e9f996197414d24b783bd100f720103` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:26.510826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c687a-66d3-4c88-b64b-746bb2f451bd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:26.512359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7374d5-b32b-4f7f-88e7-90840d3d27b6 None admin] RBAC: Authorizing `identity:get_project(project_id=3e9f996197414d24b783bd100f720103)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:26.514014 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7374d5-b32b-4f7f-88e7-90840d3d27b6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:26.516230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 206/412] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:26 2022] GET /identity/v3/projects/3e9f996197414d24b783bd100f720103 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:26.522259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7374d5-b32b-4f7f-88e7-90840d3d27b6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:26.588007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20862662-4b69-429c-be41-f0255f5d48fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:26.588574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20862662-4b69-429c-be41-f0255f5d48fe None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:26.588858 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20862662-4b69-429c-be41-f0255f5d48fe None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:26.589134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20862662-4b69-429c-be41-f0255f5d48fe None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:26.590281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20862662-4b69-429c-be41-f0255f5d48fe None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:26.591911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20862662-4b69-429c-be41-f0255f5d48fe None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:26.593964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 207/413] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:26 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:26.602234 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20862662-4b69-429c-be41-f0255f5d48fe None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:26.610231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:26.659428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7374d5-b32b-4f7f-88e7-90840d3d27b6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:26.714967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a3ba1-e6eb-4196-8ed6-7fe3a7eb1ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:26.716237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3ba1-e6eb-4196-8ed6-7fe3a7eb1ac8 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:26.717244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3ba1-e6eb-4196-8ed6-7fe3a7eb1ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:26.717737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3ba1-e6eb-4196-8ed6-7fe3a7eb1ac8 None admin] PATH_INFO: `/v3/projects/3e9f996197414d24b783bd100f720103/groups/3b936a05b35d45109e811fc637afa7df/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:26.720067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a3ba1-e6eb-4196-8ed6-7fe3a7eb1ac8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3e9f996197414d24b783bd100f720103, group_id=3b936a05b35d45109e811fc637afa7df, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:26.721751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a3ba1-e6eb-4196-8ed6-7fe3a7eb1ac8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:26.738016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 207/414] 10.222.0.86 () {58 vars in 1442 bytes} [Mon Aug 29 18:48:26 2022] PUT /identity/v3/projects/3e9f996197414d24b783bd100f720103/groups/3b936a05b35d45109e811fc637afa7df/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:48:26.792316 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:26.810810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a73a-7c99-474f-b361-24779528bc1a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:26.811367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a73a-7c99-474f-b361-24779528bc1a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:26.811367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a73a-7c99-474f-b361-24779528bc1a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:26.813041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 208/415] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:48:26.821193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728411a-3edf-4f51-b367-eaa917edc391 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:26.821526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728411a-3edf-4f51-b367-eaa917edc391 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:26.821744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728411a-3edf-4f51-b367-eaa917edc391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:26.846793 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c728411a-3edf-4f51-b367-eaa917edc391 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:26.847974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c728411a-3edf-4f51-b367-eaa917edc391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:26.871138 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20862662-4b69-429c-be41-f0255f5d48fe None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:26.945844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec01da-d6ad-4130-9f8c-0d6e161edd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:26.946524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec01da-d6ad-4130-9f8c-0d6e161edd65 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:26.946807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec01da-d6ad-4130-9f8c-0d6e161edd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:26.947107 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec01da-d6ad-4130-9f8c-0d6e161edd65 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db/groups/1865f727726848c8bad00c83f6f673eb/roles/31cc9f3a35594d40924277f4778c9f03` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:26.952539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec01da-d6ad-4130-9f8c-0d6e161edd65 None admin] RBAC: Authorizing `identity:create_grant(project_id=f456bff4d42f4ff194d2ac607af555db, group_id=1865f727726848c8bad00c83f6f673eb, role_id=31cc9f3a35594d40924277f4778c9f03)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:26.954221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec01da-d6ad-4130-9f8c-0d6e161edd65 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:26.964800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 209/416] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:26.973485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 208/417] 10.222.0.86 () {58 vars in 1442 bytes} [Mon Aug 29 18:48:26 2022] PUT /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db/groups/1865f727726848c8bad00c83f6f673eb/roles/31cc9f3a35594d40924277f4778c9f03 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:48:27.273130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728411a-3edf-4f51-b367-eaa917edc391 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:27.335496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:27.349336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd494b4-d696-40c1-9452-4e2cd5f246c8 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:27.349708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd494b4-d696-40c1-9452-4e2cd5f246c8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:27.349985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd494b4-d696-40c1-9452-4e2cd5f246c8 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:27.351752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 209/418] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:27.357076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e78d9f-1e34-4e18-aea5-d381b5a63e0e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:27.357518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e78d9f-1e34-4e18-aea5-d381b5a63e0e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:27.358046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e78d9f-1e34-4e18-aea5-d381b5a63e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:27.376390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-54e78d9f-1e34-4e18-aea5-d381b5a63e0e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:27.377731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-54e78d9f-1e34-4e18-aea5-d381b5a63e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:27.386530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728411a-3edf-4f51-b367-eaa917edc391 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:27.426565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773ad29-18f6-4adb-a0c8-17c3c5d2529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:27.427236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ad29-18f6-4adb-a0c8-17c3c5d2529e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:27.427527 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ad29-18f6-4adb-a0c8-17c3c5d2529e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:27.427852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ad29-18f6-4adb-a0c8-17c3c5d2529e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:27.439762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773ad29-18f6-4adb-a0c8-17c3c5d2529e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d773ad29-18f6-4adb-a0c8-17c3c5d2529e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:27.445067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:27.446178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:27.447057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 210/419] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:27.453759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773ad29-18f6-4adb-a0c8-17c3c5d2529e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:27.550616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 210/420] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:27.566920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773ad29-18f6-4adb-a0c8-17c3c5d2529e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:27.599420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:27.604459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3edf08-55a3-4904-9ca2-2be07a88a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:27.605247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3edf08-55a3-4904-9ca2-2be07a88a0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:27.605523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3edf08-55a3-4904-9ca2-2be07a88a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:27.605793 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3edf08-55a3-4904-9ca2-2be07a88a0e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:27.606461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3edf08-55a3-4904-9ca2-2be07a88a0e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:27.606762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3edf08-55a3-4904-9ca2-2be07a88a0e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:27.608646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3edf08-55a3-4904-9ca2-2be07a88a0e7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:27.621005 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 211/421] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:27.624144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e78d9f-1e34-4e18-aea5-d381b5a63e0e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:27.712632 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:27.777125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e78d9f-1e34-4e18-aea5-d381b5a63e0e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:27.832785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09edd26-bd52-4a3b-bca1-f444b8340a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:27.833318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09edd26-bd52-4a3b-bca1-f444b8340a77 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:27.833596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09edd26-bd52-4a3b-bca1-f444b8340a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:27.833869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09edd26-bd52-4a3b-bca1-f444b8340a77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:27.934637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09edd26-bd52-4a3b-bca1-f444b8340a77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:27.936685 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09edd26-bd52-4a3b-bca1-f444b8340a77 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:27.958083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3edf08-55a3-4904-9ca2-2be07a88a0e7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:28.019374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:28.069670 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 211/422] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:48:28.080744 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09edd26-bd52-4a3b-bca1-f444b8340a77 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:28.127984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3edf08-55a3-4904-9ca2-2be07a88a0e7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:28.191341 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432f94c-690e-48b0-be20-e579b4e3de69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:28.191984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432f94c-690e-48b0-be20-e579b4e3de69 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:28.192264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432f94c-690e-48b0-be20-e579b4e3de69 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:28.192533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432f94c-690e-48b0-be20-e579b4e3de69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:28.224189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432f94c-690e-48b0-be20-e579b4e3de69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3432f94c-690e-48b0-be20-e579b4e3de69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:28.229553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:28.230459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:28.231372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 212/423] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:48:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:28.237829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c68f75-9026-4faa-b7bf-8e1cfacb8dc2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:28.238118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c68f75-9026-4faa-b7bf-8e1cfacb8dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:28.238535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c68f75-9026-4faa-b7bf-8e1cfacb8dc2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:28.240630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 213/424] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:28.241184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c68f75-9026-4faa-b7bf-8e1cfacb8dc2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:28.257195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09edd26-bd52-4a3b-bca1-f444b8340a77 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:28.289956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:28.294423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4a6b7-9826-49e5-9717-cf0dcb0d96a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:28.295754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4a6b7-9826-49e5-9717-cf0dcb0d96a2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:28.296033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4a6b7-9826-49e5-9717-cf0dcb0d96a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:28.296304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4a6b7-9826-49e5-9717-cf0dcb0d96a2 None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:28.297439 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4a6b7-9826-49e5-9717-cf0dcb0d96a2 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:28.298890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4a6b7-9826-49e5-9717-cf0dcb0d96a2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:28.300714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 212/425] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:28 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:28.306036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fdd38-0bd2-4793-b8fd-47d77a71bf4f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:28.306329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fdd38-0bd2-4793-b8fd-47d77a71bf4f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:28.306611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fdd38-0bd2-4793-b8fd-47d77a71bf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:28.324686 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-655fdd38-0bd2-4793-b8fd-47d77a71bf4f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:28.325885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-655fdd38-0bd2-4793-b8fd-47d77a71bf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:28.343655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:28.367417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c68f75-9026-4faa-b7bf-8e1cfacb8dc2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:28.409043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 213/426] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:28.409783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fdd38-0bd2-4793-b8fd-47d77a71bf4f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:28.416626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecca0fa-df3b-42ce-a435-2954468e987b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:28.417243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecca0fa-df3b-42ce-a435-2954468e987b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:28.417518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecca0fa-df3b-42ce-a435-2954468e987b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:28.421117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecca0fa-df3b-42ce-a435-2954468e987b None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:28.421799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecca0fa-df3b-42ce-a435-2954468e987b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:28.422107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecca0fa-df3b-42ce-a435-2954468e987b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:28.423701 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecca0fa-df3b-42ce-a435-2954468e987b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:28.442055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 214/427] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:28.448844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecca0fa-df3b-42ce-a435-2954468e987b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:28.591892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fdd38-0bd2-4793-b8fd-47d77a71bf4f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:28.622725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:28.627036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecca0fa-df3b-42ce-a435-2954468e987b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:28.640649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1988050-e724-4ad0-a573-6de431f348fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:28.641287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1988050-e724-4ad0-a573-6de431f348fd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:28.641563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1988050-e724-4ad0-a573-6de431f348fd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:28.641833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1988050-e724-4ad0-a573-6de431f348fd None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:28.643113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1988050-e724-4ad0-a573-6de431f348fd None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:28.644650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1988050-e724-4ad0-a573-6de431f348fd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:28.646532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 214/428] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:28 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:28.661434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1988050-e724-4ad0-a573-6de431f348fd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:28.679247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec1427-4dde-4fc0-8f18-c8fb257cb18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:28.679858 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec1427-4dde-4fc0-8f18-c8fb257cb18b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:28.680136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec1427-4dde-4fc0-8f18-c8fb257cb18b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:28.680404 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec1427-4dde-4fc0-8f18-c8fb257cb18b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:28.737741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:28.761029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec1427-4dde-4fc0-8f18-c8fb257cb18b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:28.763258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec1427-4dde-4fc0-8f18-c8fb257cb18b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:28.827466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1988050-e724-4ad0-a573-6de431f348fd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:28.864289 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b6e32-2df7-4000-9994-d9256bda7b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:28.865050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b6e32-2df7-4000-9994-d9256bda7b69 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:28.865425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b6e32-2df7-4000-9994-d9256bda7b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:28.865802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b6e32-2df7-4000-9994-d9256bda7b69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:28.866719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b6e32-2df7-4000-9994-d9256bda7b69 None admin] RBAC: Adding query filter params (group.id=1865f727726848c8bad00c83f6f673eb, role.id=a0db5bd603fc4096917bbcd8c6932bc0, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:28.867269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b6e32-2df7-4000-9994-d9256bda7b69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:28.868796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b6e32-2df7-4000-9994-d9256bda7b69 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:28.876267 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 215/429] 10.222.0.86 () {56 vars in 1376 bytes} [Mon Aug 29 18:48:28 2022] GET /identity/v3/role_assignments?group.id=1865f727726848c8bad00c83f6f673eb&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 611 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:28.877262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b6e32-2df7-4000-9994-d9256bda7b69 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:28.896417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:28.907335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 215/430] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:28.913273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec1427-4dde-4fc0-8f18-c8fb257cb18b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:29.053406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:29.086010 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b6e32-2df7-4000-9994-d9256bda7b69 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:29.110628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec1427-4dde-4fc0-8f18-c8fb257cb18b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:29.143721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86207793-0b61-4294-ba44-790a35d3abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:29.144390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86207793-0b61-4294-ba44-790a35d3abeb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:29.144641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86207793-0b61-4294-ba44-790a35d3abeb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:29.144914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86207793-0b61-4294-ba44-790a35d3abeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:29.153455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769fe20-a719-43d6-b567-825e7ea35dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:29.155063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769fe20-a719-43d6-b567-825e7ea35dad None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:29.155826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769fe20-a719-43d6-b567-825e7ea35dad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:29.156376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769fe20-a719-43d6-b567-825e7ea35dad None admin] PATH_INFO: `/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:29.158103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769fe20-a719-43d6-b567-825e7ea35dad None admin] RBAC: Authorizing `identity:get_project(project_id=cdbff3dc80ab4433ba8d78b52eb66086)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:29.158231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86207793-0b61-4294-ba44-790a35d3abeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:29.159912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769fe20-a719-43d6-b567-825e7ea35dad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:29.161753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 216/431] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:28 2022] GET /identity/v3/projects/cdbff3dc80ab4433ba8d78b52eb66086 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:29.164377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e4ab1-5d89-4825-9833-1ca795f9b15d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:29.164618 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e4ab1-5d89-4825-9833-1ca795f9b15d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:29.164890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e4ab1-5d89-4825-9833-1ca795f9b15d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:29.169508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 217/432] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:29.169508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e4ab1-5d89-4825-9833-1ca795f9b15d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-86207793-0b61-4294-ba44-790a35d3abeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:29.170533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:29.171663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:29.172460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 216/433] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:29.179854 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb8814-f9fc-4180-a97c-c60681bb0187 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:29.180138 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb8814-f9fc-4180-a97c-c60681bb0187 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:29.180413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb8814-f9fc-4180-a97c-c60681bb0187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:29.200354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e7fb8814-f9fc-4180-a97c-c60681bb0187 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:29.201750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e7fb8814-f9fc-4180-a97c-c60681bb0187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:29.259557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:29.287093 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 217/434] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:29.288763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb8814-f9fc-4180-a97c-c60681bb0187 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:29.307755 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e4ab1-5d89-4825-9833-1ca795f9b15d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:29.348935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4f9f5-327b-46d7-979a-49e3071fdbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:29.349949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4f9f5-327b-46d7-979a-49e3071fdbc7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:29.350428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4f9f5-327b-46d7-979a-49e3071fdbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:29.350907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4f9f5-327b-46d7-979a-49e3071fdbc7 None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:29.352271 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4f9f5-327b-46d7-979a-49e3071fdbc7 None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:29.353930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4f9f5-327b-46d7-979a-49e3071fdbc7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:29.356060 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 218/435] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:29 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:29.365142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4f9f5-327b-46d7-979a-49e3071fdbc7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:29.391914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:29.428138 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb8814-f9fc-4180-a97c-c60681bb0187 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:29.476071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bdaf9-d09d-4865-ab9f-b2132a71b39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:29.476702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bdaf9-d09d-4865-ab9f-b2132a71b39f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:29.476984 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bdaf9-d09d-4865-ab9f-b2132a71b39f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:29.477259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bdaf9-d09d-4865-ab9f-b2132a71b39f None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:29.477921 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bdaf9-d09d-4865-ab9f-b2132a71b39f None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:29.478221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bdaf9-d09d-4865-ab9f-b2132a71b39f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:29.480958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bdaf9-d09d-4865-ab9f-b2132a71b39f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:29.490605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 218/436] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:48:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:29.491713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26badfb3-d428-43d8-ad64-e5408503b82a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:29.492009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26badfb3-d428-43d8-ad64-e5408503b82a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:29.492508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26badfb3-d428-43d8-ad64-e5408503b82a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:29.495556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 219/437] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:29.500693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be6bf2-727c-4ad1-8bb0-2de4522e4c41 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:29.501070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be6bf2-727c-4ad1-8bb0-2de4522e4c41 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:29.501309 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be6bf2-727c-4ad1-8bb0-2de4522e4c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:29.504580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4f9f5-327b-46d7-979a-49e3071fdbc7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:29.523558 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-16be6bf2-727c-4ad1-8bb0-2de4522e4c41 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:29.525145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-16be6bf2-727c-4ad1-8bb0-2de4522e4c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:29.550529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e11949-c67d-494a-b49a-4208b798c369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:29.551180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e11949-c67d-494a-b49a-4208b798c369 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:29.551458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e11949-c67d-494a-b49a-4208b798c369 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:29.551727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e11949-c67d-494a-b49a-4208b798c369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:29.552448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e11949-c67d-494a-b49a-4208b798c369 None admin] RBAC: Adding query filter params (group.id=1865f727726848c8bad00c83f6f673eb, role.id=31cc9f3a35594d40924277f4778c9f03, scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:29.552751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e11949-c67d-494a-b49a-4208b798c369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:29.554171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e11949-c67d-494a-b49a-4208b798c369 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:29.558663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:29.563032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 219/438] 10.222.0.86 () {56 vars in 1376 bytes} [Mon Aug 29 18:48:29 2022] GET /identity/v3/role_assignments?group.id=1865f727726848c8bad00c83f6f673eb&scope.project.id=cdbff3dc80ab4433ba8d78b52eb66086&role.id=31cc9f3a35594d40924277f4778c9f03 => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:29.563032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e11949-c67d-494a-b49a-4208b798c369 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:29.612549 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 220/439] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:29.613422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be6bf2-727c-4ad1-8bb0-2de4522e4c41 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:29.713718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e11949-c67d-494a-b49a-4208b798c369 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:29.721952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:29.758162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8e843-ed5b-4485-a062-ff0cad1e1071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:29.758869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8e843-ed5b-4485-a062-ff0cad1e1071 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:29.759251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8e843-ed5b-4485-a062-ff0cad1e1071 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:29.759522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8e843-ed5b-4485-a062-ff0cad1e1071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:29.761595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be6bf2-727c-4ad1-8bb0-2de4522e4c41 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:29.812381 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cff6b-b0be-41b5-a7c0-4fb2c1fddb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:29.813017 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cff6b-b0be-41b5-a7c0-4fb2c1fddb08 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:29.813404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cff6b-b0be-41b5-a7c0-4fb2c1fddb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:29.813684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cff6b-b0be-41b5-a7c0-4fb2c1fddb08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:29.825232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cff6b-b0be-41b5-a7c0-4fb2c1fddb08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9e6cff6b-b0be-41b5-a7c0-4fb2c1fddb08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:29.830649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:29.831682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:29.832779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 221/440] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:29.837321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cff6b-b0be-41b5-a7c0-4fb2c1fddb08 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:29.850693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:29.854863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8e843-ed5b-4485-a062-ff0cad1e1071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:29.858160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8e843-ed5b-4485-a062-ff0cad1e1071 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:29.965330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cff6b-b0be-41b5-a7c0-4fb2c1fddb08 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:29.982092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 220/441] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:29.983098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8e843-ed5b-4485-a062-ff0cad1e1071 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:29.999845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:30.009049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03cd76-4161-4b1b-82d8-05e869086944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:30.009659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03cd76-4161-4b1b-82d8-05e869086944 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:30.009937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03cd76-4161-4b1b-82d8-05e869086944 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:30.010208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03cd76-4161-4b1b-82d8-05e869086944 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:30.010868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03cd76-4161-4b1b-82d8-05e869086944 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:30.011198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03cd76-4161-4b1b-82d8-05e869086944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:30.012558 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03cd76-4161-4b1b-82d8-05e869086944 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:30.021255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 222/442] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:30.022141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03cd76-4161-4b1b-82d8-05e869086944 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:30.086193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:30.090054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8e843-ed5b-4485-a062-ff0cad1e1071 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:30.120103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03cd76-4161-4b1b-82d8-05e869086944 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:30.125158 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83651428-ba40-4999-860e-7f66209c2ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:30.125870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83651428-ba40-4999-860e-7f66209c2ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:30.126146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83651428-ba40-4999-860e-7f66209c2ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:30.126436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83651428-ba40-4999-860e-7f66209c2ad3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:30.136485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83651428-ba40-4999-860e-7f66209c2ad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-83651428-ba40-4999-860e-7f66209c2ad3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:30.141430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:30.142348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:30.143266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 221/443] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:48:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:30.143940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83651428-ba40-4999-860e-7f66209c2ad3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:30.154286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bda37-ffd2-424c-8a78-565e46e943e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:30.154914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bda37-ffd2-424c-8a78-565e46e943e0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:30.155218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bda37-ffd2-424c-8a78-565e46e943e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:30.155494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bda37-ffd2-424c-8a78-565e46e943e0 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:30.156671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bda37-ffd2-424c-8a78-565e46e943e0 None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:30.158111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bda37-ffd2-424c-8a78-565e46e943e0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:30.160067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 223/444] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:30 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:30.160815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bda37-ffd2-424c-8a78-565e46e943e0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:30.217162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:30.250447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83651428-ba40-4999-860e-7f66209c2ad3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:30.261284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bda37-ffd2-424c-8a78-565e46e943e0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:30.283607 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60e10-6652-4332-be5b-4dfb9232c3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:30.284211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60e10-6652-4332-be5b-4dfb9232c3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:30.284487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60e10-6652-4332-be5b-4dfb9232c3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:30.284757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60e10-6652-4332-be5b-4dfb9232c3b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:30.293910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e224e-2741-43fa-93c7-ea6c9afd9735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:30.294504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e224e-2741-43fa-93c7-ea6c9afd9735 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:30.295318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e224e-2741-43fa-93c7-ea6c9afd9735 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:30.295600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e224e-2741-43fa-93c7-ea6c9afd9735 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:30.296254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e224e-2741-43fa-93c7-ea6c9afd9735 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:30.296554 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e224e-2741-43fa-93c7-ea6c9afd9735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:30.297908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e224e-2741-43fa-93c7-ea6c9afd9735 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:30.306437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 224/445] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:30.307243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e224e-2741-43fa-93c7-ea6c9afd9735 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:30.314778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:30.355353 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a60e10-6652-4332-be5b-4dfb9232c3b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:30.356824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a60e10-6652-4332-be5b-4dfb9232c3b3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:30.404722 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e224e-2741-43fa-93c7-ea6c9afd9735 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:30.421738 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:30.440128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee3af2-0ded-4062-889c-d57ce35dda1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:30.440746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3af2-0ded-4062-889c-d57ce35dda1e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:30.441022 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3af2-0ded-4062-889c-d57ce35dda1e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:30.441293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3af2-0ded-4062-889c-d57ce35dda1e None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:30.442385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee3af2-0ded-4062-889c-d57ce35dda1e None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:30.444110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee3af2-0ded-4062-889c-d57ce35dda1e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:30.445651 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 225/446] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:30 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:30.446401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee3af2-0ded-4062-889c-d57ce35dda1e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:30.454589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 222/447] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:30.455458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60e10-6652-4332-be5b-4dfb9232c3b3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:30.512352 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:30.550841 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee3af2-0ded-4062-889c-d57ce35dda1e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:30.552661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60e10-6652-4332-be5b-4dfb9232c3b3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:30.584120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f773651-c2b1-4a36-abbe-c1f843efe916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:30.585059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f773651-c2b1-4a36-abbe-c1f843efe916 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:30.585338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f773651-c2b1-4a36-abbe-c1f843efe916 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:30.585459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c7a83-72d2-4025-9404-49a5882f8bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:30.585613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f773651-c2b1-4a36-abbe-c1f843efe916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:30.585950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c7a83-72d2-4025-9404-49a5882f8bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:30.586230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c7a83-72d2-4025-9404-49a5882f8bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:30.586502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c7a83-72d2-4025-9404-49a5882f8bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:30.587246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c7a83-72d2-4025-9404-49a5882f8bf2 None admin] RBAC: Adding query filter params (group.id=1865f727726848c8bad00c83f6f673eb, role.id=a0db5bd603fc4096917bbcd8c6932bc0, scope.project.id=f456bff4d42f4ff194d2ac607af555db) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:30.587554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c7a83-72d2-4025-9404-49a5882f8bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:30.589013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c7a83-72d2-4025-9404-49a5882f8bf2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:30.595203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 223/448] 10.222.0.86 () {56 vars in 1376 bytes} [Mon Aug 29 18:48:30 2022] GET /identity/v3/role_assignments?group.id=1865f727726848c8bad00c83f6f673eb&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=a0db5bd603fc4096917bbcd8c6932bc0 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:30.596048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c7a83-72d2-4025-9404-49a5882f8bf2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:30.661229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:30.681232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f773651-c2b1-4a36-abbe-c1f843efe916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:30.683306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f773651-c2b1-4a36-abbe-c1f843efe916 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:30.718504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c7a83-72d2-4025-9404-49a5882f8bf2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:30.745678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:30.759584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2de0d8-e9fb-4087-ac4a-d6904e8e9615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:30.760205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2de0d8-e9fb-4087-ac4a-d6904e8e9615 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:30.760482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2de0d8-e9fb-4087-ac4a-d6904e8e9615 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:30.760751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2de0d8-e9fb-4087-ac4a-d6904e8e9615 None admin] PATH_INFO: `/v3/projects/3e9f996197414d24b783bd100f720103` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:30.762584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2de0d8-e9fb-4087-ac4a-d6904e8e9615 None admin] RBAC: Authorizing `identity:get_project(project_id=3e9f996197414d24b783bd100f720103)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:30.763379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2de0d8-e9fb-4087-ac4a-d6904e8e9615 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:30.765065 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 224/449] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:30 2022] GET /identity/v3/projects/3e9f996197414d24b783bd100f720103 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:30.771088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2de0d8-e9fb-4087-ac4a-d6904e8e9615 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:30.813812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 226/450] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:30.819502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f773651-c2b1-4a36-abbe-c1f843efe916 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:30.903225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2de0d8-e9fb-4087-ac4a-d6904e8e9615 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:30.918825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:30.937044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f773651-c2b1-4a36-abbe-c1f843efe916 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:30.948332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021375d-a12d-4ecf-aaa2-de15d8323172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:30.949191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021375d-a12d-4ecf-aaa2-de15d8323172 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:30.949323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021375d-a12d-4ecf-aaa2-de15d8323172 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:30.949506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021375d-a12d-4ecf-aaa2-de15d8323172 None admin] PATH_INFO: `/v3/groups/3b936a05b35d45109e811fc637afa7df` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:30.950624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021375d-a12d-4ecf-aaa2-de15d8323172 None admin] RBAC: Authorizing `identity:get_group(group_id=3b936a05b35d45109e811fc637afa7df)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:30.952660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021375d-a12d-4ecf-aaa2-de15d8323172 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:30.954413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 225/451] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:30 2022] GET /identity/v3/groups/3b936a05b35d45109e811fc637afa7df => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:30.960358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021375d-a12d-4ecf-aaa2-de15d8323172 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:30.976838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402423c-5c61-406b-9645-96ca142e6539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:30.977467 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402423c-5c61-406b-9645-96ca142e6539 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:30.977749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402423c-5c61-406b-9645-96ca142e6539 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:30.978024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402423c-5c61-406b-9645-96ca142e6539 None admin] PATH_INFO: `/v3/projects/f456bff4d42f4ff194d2ac607af555db` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:30.979160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402423c-5c61-406b-9645-96ca142e6539 None admin] RBAC: Authorizing `identity:get_project(project_id=f456bff4d42f4ff194d2ac607af555db)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:30.980591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402423c-5c61-406b-9645-96ca142e6539 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:30.982431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 227/452] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:48:30 2022] GET /identity/v3/projects/f456bff4d42f4ff194d2ac607af555db => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:30.987968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402423c-5c61-406b-9645-96ca142e6539 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:31.020074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:31.061829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021375d-a12d-4ecf-aaa2-de15d8323172 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:31.087829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402423c-5c61-406b-9645-96ca142e6539 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:31.096594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767735d-3e5a-4ad0-aa64-d554be8e7bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:31.097198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767735d-3e5a-4ad0-aa64-d554be8e7bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:31.097480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767735d-3e5a-4ad0-aa64-d554be8e7bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:31.097755 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767735d-3e5a-4ad0-aa64-d554be8e7bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:31.098476 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767735d-3e5a-4ad0-aa64-d554be8e7bf2 None admin] RBAC: Adding query filter params (group.id=3b936a05b35d45109e811fc637afa7df, role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=3e9f996197414d24b783bd100f720103) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:31.098782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767735d-3e5a-4ad0-aa64-d554be8e7bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:31.100253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767735d-3e5a-4ad0-aa64-d554be8e7bf2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:31.106288 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 226/453] 10.222.0.86 () {56 vars in 1376 bytes} [Mon Aug 29 18:48:30 2022] GET /identity/v3/role_assignments?group.id=3b936a05b35d45109e811fc637afa7df&scope.project.id=3e9f996197414d24b783bd100f720103&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:31.122456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183a7bb-90ab-468e-9423-2a896957f0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:31.123140 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183a7bb-90ab-468e-9423-2a896957f0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:31.123425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183a7bb-90ab-468e-9423-2a896957f0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:31.123697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183a7bb-90ab-468e-9423-2a896957f0d6 None admin] PATH_INFO: `/v3/groups/1865f727726848c8bad00c83f6f673eb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:31.124891 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183a7bb-90ab-468e-9423-2a896957f0d6 None admin] RBAC: Authorizing `identity:get_group(group_id=1865f727726848c8bad00c83f6f673eb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:31.126494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183a7bb-90ab-468e-9423-2a896957f0d6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:31.128227 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 228/454] 10.222.0.86 () {56 vars in 1180 bytes} [Mon Aug 29 18:48:30 2022] GET /identity/v3/groups/1865f727726848c8bad00c83f6f673eb => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:31.134652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767735d-3e5a-4ad0-aa64-d554be8e7bf2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:31.238514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767735d-3e5a-4ad0-aa64-d554be8e7bf2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:31.262191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:31.273781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24203e99-8b7a-46fe-b1e6-982e0b5fd575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:31.274315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24203e99-8b7a-46fe-b1e6-982e0b5fd575 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:31.274596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24203e99-8b7a-46fe-b1e6-982e0b5fd575 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:31.274868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24203e99-8b7a-46fe-b1e6-982e0b5fd575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:31.275615 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24203e99-8b7a-46fe-b1e6-982e0b5fd575 None admin] RBAC: Adding query filter params (group.id=1865f727726848c8bad00c83f6f673eb, role.id=31cc9f3a35594d40924277f4778c9f03, scope.project.id=f456bff4d42f4ff194d2ac607af555db) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:31.275924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24203e99-8b7a-46fe-b1e6-982e0b5fd575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:31.277402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24203e99-8b7a-46fe-b1e6-982e0b5fd575 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:31.283245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 227/455] 10.222.0.86 () {56 vars in 1376 bytes} [Mon Aug 29 18:48:31 2022] GET /identity/v3/role_assignments?group.id=1865f727726848c8bad00c83f6f673eb&scope.project.id=f456bff4d42f4ff194d2ac607af555db&role.id=31cc9f3a35594d40924277f4778c9f03 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:33.863092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323658a-879e-413b-a767-86c0c15f3ab5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:33.864030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323658a-879e-413b-a767-86c0c15f3ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:33.864515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323658a-879e-413b-a767-86c0c15f3ab5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:33.866048 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 229/456] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:33.871417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28db89-1d3d-494d-87d4-e9c487df98b3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:33.872234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28db89-1d3d-494d-87d4-e9c487df98b3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:33.872714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28db89-1d3d-494d-87d4-e9c487df98b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:33.876681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515357dc-cbbe-4921-86a6-932df675a541 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:33.877194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515357dc-cbbe-4921-86a6-932df675a541 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:33.877654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515357dc-cbbe-4921-86a6-932df675a541 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:33.879397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 230/457] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:48:33.884543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e8f20-a8ba-436b-947b-654a8cb4ea20 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:33.885049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e8f20-a8ba-436b-947b-654a8cb4ea20 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:33.885511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e8f20-a8ba-436b-947b-654a8cb4ea20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:33.913870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-579e8f20-a8ba-436b-947b-654a8cb4ea20 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:33.915440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-579e8f20-a8ba-436b-947b-654a8cb4ea20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:33.916558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bd28db89-1d3d-494d-87d4-e9c487df98b3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:33.919876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bd28db89-1d3d-494d-87d4-e9c487df98b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:33.947538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:34.021070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 231/458] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:34.022601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 228/459] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:34.057301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891a866-7773-4562-a2a3-26d5a626cafc None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:34.057811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891a866-7773-4562-a2a3-26d5a626cafc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:34.058225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891a866-7773-4562-a2a3-26d5a626cafc None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:34.059764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 232/460] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:34.065214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba1219-534d-431d-ac1a-19a2e263c798 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:34.065786 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba1219-534d-431d-ac1a-19a2e263c798 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:34.066258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba1219-534d-431d-ac1a-19a2e263c798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:34.085811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c0ba1219-534d-431d-ac1a-19a2e263c798 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:34.087311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c0ba1219-534d-431d-ac1a-19a2e263c798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:34.162714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 229/461] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:34.321324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891a866-7773-4562-a2a3-26d5a626cafc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:34.343439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba1219-534d-431d-ac1a-19a2e263c798 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:34.441384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891a866-7773-4562-a2a3-26d5a626cafc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:34.472845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:34.482460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4711d58-663e-44ac-b7d2-82cf620c77ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:34.483390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4711d58-663e-44ac-b7d2-82cf620c77ca None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:34.483819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4711d58-663e-44ac-b7d2-82cf620c77ca None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:34.484238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4711d58-663e-44ac-b7d2-82cf620c77ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:34.495833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4711d58-663e-44ac-b7d2-82cf620c77ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:34.498218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4711d58-663e-44ac-b7d2-82cf620c77ca None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c4711d58-663e-44ac-b7d2-82cf620c77ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:34.515864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:34.517009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:34.518130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:34.518130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:34.518130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:34.520934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 233/462] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:34.521904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4711d58-663e-44ac-b7d2-82cf620c77ca None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:34.526536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:34.581447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba1219-534d-431d-ac1a-19a2e263c798 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:34.617146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf518f6-b9c6-4629-ae88-48f7086f1ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:34.617742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf518f6-b9c6-4629-ae88-48f7086f1ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:34.618021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf518f6-b9c6-4629-ae88-48f7086f1ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:34.618291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf518f6-b9c6-4629-ae88-48f7086f1ab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:34.629753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf518f6-b9c6-4629-ae88-48f7086f1ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:34.631335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf518f6-b9c6-4629-ae88-48f7086f1ab1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-edf518f6-b9c6-4629-ae88-48f7086f1ab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:34.650830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:34.652035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:34.652960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:34.652960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:34.652960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:34.652960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 230/463] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:34.653186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf518f6-b9c6-4629-ae88-48f7086f1ab1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:34.662423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4711d58-663e-44ac-b7d2-82cf620c77ca None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:34.711101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325b30f-3711-4e1f-9078-e006311402dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:34.712340 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325b30f-3711-4e1f-9078-e006311402dc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:34.712766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325b30f-3711-4e1f-9078-e006311402dc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:34.713177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325b30f-3711-4e1f-9078-e006311402dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:34.728040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325b30f-3711-4e1f-9078-e006311402dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:34.730560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325b30f-3711-4e1f-9078-e006311402dc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6325b30f-3711-4e1f-9078-e006311402dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:34.751225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:34.752347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:34.753379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:34.753379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:34.753379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:34.755873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 234/464] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:34.756869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325b30f-3711-4e1f-9078-e006311402dc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:34.769748 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:34.772885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf518f6-b9c6-4629-ae88-48f7086f1ab1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:34.816593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeef27-44fd-4d6b-8d70-c4b281ec2fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:34.817708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeef27-44fd-4d6b-8d70-c4b281ec2fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:34.818149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeef27-44fd-4d6b-8d70-c4b281ec2fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:34.818825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeef27-44fd-4d6b-8d70-c4b281ec2fd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:34.819876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeef27-44fd-4d6b-8d70-c4b281ec2fd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:34.820772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeef27-44fd-4d6b-8d70-c4b281ec2fd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:34.822562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeef27-44fd-4d6b-8d70-c4b281ec2fd2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:34.831102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:34.845786 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 231/465] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:34.847037 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeef27-44fd-4d6b-8d70-c4b281ec2fd2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:34.891585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325b30f-3711-4e1f-9078-e006311402dc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:34.930945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8da597-48a4-4550-b541-f32c1f338e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:34.931576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8da597-48a4-4550-b541-f32c1f338e78 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:34.931857 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8da597-48a4-4550-b541-f32c1f338e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:34.932137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8da597-48a4-4550-b541-f32c1f338e78 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:34.932894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8da597-48a4-4550-b541-f32c1f338e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:34.933201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8da597-48a4-4550-b541-f32c1f338e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:34.934560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8da597-48a4-4550-b541-f32c1f338e78 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:34.947899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 235/466] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:34.948517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8da597-48a4-4550-b541-f32c1f338e78 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:34.956856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeef27-44fd-4d6b-8d70-c4b281ec2fd2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:34.995787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91048a1d-5b7b-467c-8039-d53ccbd8a01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:34.996665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91048a1d-5b7b-467c-8039-d53ccbd8a01f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:34.996951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91048a1d-5b7b-467c-8039-d53ccbd8a01f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:34.997221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91048a1d-5b7b-467c-8039-d53ccbd8a01f None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:34.998055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91048a1d-5b7b-467c-8039-d53ccbd8a01f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:34.998732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91048a1d-5b7b-467c-8039-d53ccbd8a01f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:35.000263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91048a1d-5b7b-467c-8039-d53ccbd8a01f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:35.012486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 232/467] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:35.013869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a627-9ac6-4587-9074-958d9e634f51 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:35.014157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a627-9ac6-4587-9074-958d9e634f51 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:35.014432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a627-9ac6-4587-9074-958d9e634f51 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:35.015955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 233/468] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:35.020260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a627-9ac6-4587-9074-958d9e634f51 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:35.042639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:35.060848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8da597-48a4-4550-b541-f32c1f338e78 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:35.078629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:35.098749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749e1e5-2334-484b-baaa-2f2193a7f264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:35.099355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e1e5-2334-484b-baaa-2f2193a7f264 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:35.099633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e1e5-2334-484b-baaa-2f2193a7f264 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:35.099975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e1e5-2334-484b-baaa-2f2193a7f264 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:35.100623 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749e1e5-2334-484b-baaa-2f2193a7f264 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:35.102130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749e1e5-2334-484b-baaa-2f2193a7f264 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:35.127196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 236/469] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:48:34 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:48:35.128017 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749e1e5-2334-484b-baaa-2f2193a7f264 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:35.143614 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a627-9ac6-4587-9074-958d9e634f51 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:35.182522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97852d-fa4e-4492-a685-a317732ddf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:35.183190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97852d-fa4e-4492-a685-a317732ddf72 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:35.183474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97852d-fa4e-4492-a685-a317732ddf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:35.183749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97852d-fa4e-4492-a685-a317732ddf72 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:35.184409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97852d-fa4e-4492-a685-a317732ddf72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:35.185918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97852d-fa4e-4492-a685-a317732ddf72 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:35.209917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 234/470] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:48:35 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:48:35.211583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ac900-2504-4172-a3d2-7673bbfb1dbf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:35.212092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ac900-2504-4172-a3d2-7673bbfb1dbf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:35.212567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ac900-2504-4172-a3d2-7673bbfb1dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:35.246431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c95ac900-2504-4172-a3d2-7673bbfb1dbf None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:35.248986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c95ac900-2504-4172-a3d2-7673bbfb1dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:35.258355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749e1e5-2334-484b-baaa-2f2193a7f264 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:35.314966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d816274-c295-4626-9a28-5e00ea603512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:35.316977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d816274-c295-4626-9a28-5e00ea603512 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:35.316977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d816274-c295-4626-9a28-5e00ea603512 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:35.316977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d816274-c295-4626-9a28-5e00ea603512 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:35.317241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d816274-c295-4626-9a28-5e00ea603512 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:35.319057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d816274-c295-4626-9a28-5e00ea603512 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:35.346745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 237/471] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:48:35 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:48:35.348220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315c511-bfe5-47ee-94aa-367f2e559203 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:35.348510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315c511-bfe5-47ee-94aa-367f2e559203 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:35.348875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315c511-bfe5-47ee-94aa-367f2e559203 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:35.350179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 238/472] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:35.356225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 235/473] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:35.356416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8717ecd-9065-4eca-9bd9-6d1f307114c7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:35.356630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8717ecd-9065-4eca-9bd9-6d1f307114c7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:35.356908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8717ecd-9065-4eca-9bd9-6d1f307114c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:35.380305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d8717ecd-9065-4eca-9bd9-6d1f307114c7 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:35.381555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d8717ecd-9065-4eca-9bd9-6d1f307114c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:35.408805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:35.469120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 239/474] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:35.640063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ac900-2504-4172-a3d2-7673bbfb1dbf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:35.649094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:35.755134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8717ecd-9065-4eca-9bd9-6d1f307114c7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:35.766961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ac900-2504-4172-a3d2-7673bbfb1dbf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:35.824021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8862346-cf9f-4cbe-a5aa-91d7cbdb908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:35.824660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862346-cf9f-4cbe-a5aa-91d7cbdb908e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:35.824933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862346-cf9f-4cbe-a5aa-91d7cbdb908e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:35.825199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862346-cf9f-4cbe-a5aa-91d7cbdb908e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:35.846462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8862346-cf9f-4cbe-a5aa-91d7cbdb908e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:35.848053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8862346-cf9f-4cbe-a5aa-91d7cbdb908e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f8862346-cf9f-4cbe-a5aa-91d7cbdb908e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:35.884239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:35.885180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:35.886687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:35.886687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:35.886687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:35.886687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 236/475] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:35.895403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8862346-cf9f-4cbe-a5aa-91d7cbdb908e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:35.916447 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8717ecd-9065-4eca-9bd9-6d1f307114c7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:35.983994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:35.992918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befd1dc-9e0d-4bc4-aced-4cd4e8c4ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:35.993784 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd1dc-9e0d-4bc4-aced-4cd4e8c4ca52 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:35.994057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd1dc-9e0d-4bc4-aced-4cd4e8c4ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:35.994325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd1dc-9e0d-4bc4-aced-4cd4e8c4ca52 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:35.995024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befd1dc-9e0d-4bc4-aced-4cd4e8c4ca52 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:35.996390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befd1dc-9e0d-4bc4-aced-4cd4e8c4ca52 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:36.017403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.sql.core [None req-5befd1dc-9e0d-4bc4-aced-4cd4e8c4ca52 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 18:48:36.017403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 18:48:36.017403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [parameters: {'id': 'f61b796051d94fae848e9f6b9949cc6f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 18:48:36.017403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81499) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5befd1dc-9e0d-4bc4-aced-4cd4e8c4ca52 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application result.read() Aug 29 18:48:36.036941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return ref.to_dict() Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:48:36.037852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application yield resource Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise self.value Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application session.commit() Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 18:48:36.039423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.session.flush() Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application compat.raise_( Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 18:48:36.040270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:48:36.041216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application result.read() Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:48:36.042204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [parameters: {'id': 'f61b796051d94fae848e9f6b9949cc6f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:36.043815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 18:48:36.046420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:36.052568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 240/476] 10.222.0.86 () {60 vars in 1131 bytes} [Mon Aug 29 18:48:35 2022] POST /identity/v3/roles => generated 144 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:36.058294 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8862346-cf9f-4cbe-a5aa-91d7cbdb908e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:36.062465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befd1dc-9e0d-4bc4-aced-4cd4e8c4ca52 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:36.099617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388615a-de5e-485e-a3f6-1b6991c793db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:36.100224 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388615a-de5e-485e-a3f6-1b6991c793db None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:36.100500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388615a-de5e-485e-a3f6-1b6991c793db None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:36.100772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388615a-de5e-485e-a3f6-1b6991c793db None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:36.101414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388615a-de5e-485e-a3f6-1b6991c793db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:36.101747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388615a-de5e-485e-a3f6-1b6991c793db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:36.103159 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388615a-de5e-485e-a3f6-1b6991c793db None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:36.106029 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:36.134765 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 237/477] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:36.142045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388615a-de5e-485e-a3f6-1b6991c793db None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:36.249595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befd1dc-9e0d-4bc4-aced-4cd4e8c4ca52 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:36.275235 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388615a-de5e-485e-a3f6-1b6991c793db None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:36.310902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036581d6-6be1-4b08-9e89-fdeca1dda734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:36.313998 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036581d6-6be1-4b08-9e89-fdeca1dda734 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:36.313998 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036581d6-6be1-4b08-9e89-fdeca1dda734 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:36.313998 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036581d6-6be1-4b08-9e89-fdeca1dda734 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:36.314344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:36.334043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036581d6-6be1-4b08-9e89-fdeca1dda734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:36.334915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8a8c4-fbf8-4826-88b0-8958d70017f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:36.335759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a8c4-fbf8-4826-88b0-8958d70017f0 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:36.336230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a8c4-fbf8-4826-88b0-8958d70017f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:36.336706 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a8c4-fbf8-4826-88b0-8958d70017f0 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:36.337540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8a8c4-fbf8-4826-88b0-8958d70017f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:36.339270 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8a8c4-fbf8-4826-88b0-8958d70017f0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-036581d6-6be1-4b08-9e89-fdeca1dda734 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:36.344964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:36.345830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:48:36.346613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:36.360076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 241/478] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:48:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:36.369951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036581d6-6be1-4b08-9e89-fdeca1dda734 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:36.387572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 238/479] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:48:36 2022] POST /identity/v3/users => generated 249 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:48:36.467093 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:36.495396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036581d6-6be1-4b08-9e89-fdeca1dda734 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:36.542349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b8a0dd-a044-4247-a6f3-2cce6531a31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:36.542971 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8a0dd-a044-4247-a6f3-2cce6531a31b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:36.543270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8a0dd-a044-4247-a6f3-2cce6531a31b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:36.543536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8a0dd-a044-4247-a6f3-2cce6531a31b None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:36.544207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b8a0dd-a044-4247-a6f3-2cce6531a31b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:36.544504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b8a0dd-a044-4247-a6f3-2cce6531a31b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:36.546006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b8a0dd-a044-4247-a6f3-2cce6531a31b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:36.555027 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 242/480] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:37.637247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f295001-b86b-4629-bbdc-45b13c010fcc None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:37.638333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f295001-b86b-4629-bbdc-45b13c010fcc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:37.638815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f295001-b86b-4629-bbdc-45b13c010fcc None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:37.641594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 239/481] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:37.648042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f21299-a718-41c7-a363-7fb37ccd1df8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:37.648898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f21299-a718-41c7-a363-7fb37ccd1df8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:37.649384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f21299-a718-41c7-a363-7fb37ccd1df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:37.681459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b2f21299-a718-41c7-a363-7fb37ccd1df8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:37.689409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b2f21299-a718-41c7-a363-7fb37ccd1df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:37.784675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 243/482] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:37.797452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7016347-a3d2-460d-b06f-528569ef82bc None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:37.798086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7016347-a3d2-460d-b06f-528569ef82bc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:37.798581 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7016347-a3d2-460d-b06f-528569ef82bc None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:37.800415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 240/483] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:37.806207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516630-b823-434a-bb39-9a4702952781 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:37.806730 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516630-b823-434a-bb39-9a4702952781 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:37.807259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516630-b823-434a-bb39-9a4702952781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:37.839304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a5516630-b823-434a-bb39-9a4702952781 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:37.840848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a5516630-b823-434a-bb39-9a4702952781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:37.842977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bff71-727c-4214-b6fc-15d1c1f71f80 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:37.842977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bff71-727c-4214-b6fc-15d1c1f71f80 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:37.842977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bff71-727c-4214-b6fc-15d1c1f71f80 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:37.843449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 241/484] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:37.848195 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317667-be4f-48c2-b596-bd0a865f15de None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:37.848473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317667-be4f-48c2-b596-bd0a865f15de None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:37.848742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317667-be4f-48c2-b596-bd0a865f15de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:37.895119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ca317667-be4f-48c2-b596-bd0a865f15de None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:37.896239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ca317667-be4f-48c2-b596-bd0a865f15de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:37.921191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 244/485] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:38.035829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 242/486] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:38.209412 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5516630-b823-434a-bb39-9a4702952781 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:38.213108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:38.214863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca317667-be4f-48c2-b596-bd0a865f15de None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:38.324565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5516630-b823-434a-bb39-9a4702952781 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:38.362603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535ac18-77b7-42a1-948a-02d3beb4d4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:38.363650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ac18-77b7-42a1-948a-02d3beb4d4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:38.364123 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ac18-77b7-42a1-948a-02d3beb4d4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:38.364585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ac18-77b7-42a1-948a-02d3beb4d4c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:38.375964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535ac18-77b7-42a1-948a-02d3beb4d4c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1535ac18-77b7-42a1-948a-02d3beb4d4c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:38.381259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:38.382135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:38.383024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:38.385598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 245/487] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:38.391829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535ac18-77b7-42a1-948a-02d3beb4d4c8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:38.403087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca317667-be4f-48c2-b596-bd0a865f15de None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:38.406787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:38.442380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b0965-04cb-4a2c-970e-fbd6226be376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:38.443343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b0965-04cb-4a2c-970e-fbd6226be376 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:38.444010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b0965-04cb-4a2c-970e-fbd6226be376 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:38.444413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b0965-04cb-4a2c-970e-fbd6226be376 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:38.460033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b0965-04cb-4a2c-970e-fbd6226be376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0e2b0965-04cb-4a2c-970e-fbd6226be376 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:38.468809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:38.469830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:38.470612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:38.471257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 243/488] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:38.472316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b0965-04cb-4a2c-970e-fbd6226be376 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:38.484493 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:38.528478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535ac18-77b7-42a1-948a-02d3beb4d4c8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:38.567445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb209d-7d21-4d45-b0bb-7491267208bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:38.569150 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb209d-7d21-4d45-b0bb-7491267208bc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:38.569885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb209d-7d21-4d45-b0bb-7491267208bc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:38.570350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb209d-7d21-4d45-b0bb-7491267208bc None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:38.571474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb209d-7d21-4d45-b0bb-7491267208bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:38.572388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb209d-7d21-4d45-b0bb-7491267208bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:38.573956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb209d-7d21-4d45-b0bb-7491267208bc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:38.587327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 246/489] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:38.588514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb209d-7d21-4d45-b0bb-7491267208bc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:38.588619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b0965-04cb-4a2c-970e-fbd6226be376 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:38.628280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525f50c-3aa4-4069-8904-c19b6402bf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:38.628924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525f50c-3aa4-4069-8904-c19b6402bf6d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:38.629266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525f50c-3aa4-4069-8904-c19b6402bf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:38.629528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525f50c-3aa4-4069-8904-c19b6402bf6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:38.641495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525f50c-3aa4-4069-8904-c19b6402bf6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6525f50c-3aa4-4069-8904-c19b6402bf6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:38.646815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:38.647729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:38.648510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 244/490] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:38.649181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525f50c-3aa4-4069-8904-c19b6402bf6d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:38.695369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:38.724745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb209d-7d21-4d45-b0bb-7491267208bc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:38.756375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:38.769045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c470c-4eae-4449-9b8e-ef2c99a0969a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:38.770536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c470c-4eae-4449-9b8e-ef2c99a0969a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:38.772447 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c470c-4eae-4449-9b8e-ef2c99a0969a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:38.772733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c470c-4eae-4449-9b8e-ef2c99a0969a None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:38.773497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c470c-4eae-4449-9b8e-ef2c99a0969a None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:38.773805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c470c-4eae-4449-9b8e-ef2c99a0969a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:38.776122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c470c-4eae-4449-9b8e-ef2c99a0969a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:38.794075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 247/491] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:38.795586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c470c-4eae-4449-9b8e-ef2c99a0969a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:38.796117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525f50c-3aa4-4069-8904-c19b6402bf6d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:38.846442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233aae8-5f44-428e-ac93-025fbac66507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:38.847487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233aae8-5f44-428e-ac93-025fbac66507 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:38.847959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233aae8-5f44-428e-ac93-025fbac66507 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:38.848446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233aae8-5f44-428e-ac93-025fbac66507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:38.860314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233aae8-5f44-428e-ac93-025fbac66507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:38.862096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233aae8-5f44-428e-ac93-025fbac66507 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9233aae8-5f44-428e-ac93-025fbac66507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:38.877861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:38.878781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:38.879672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:38.879672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:38.879672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:38.881705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 245/492] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:38.882688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233aae8-5f44-428e-ac93-025fbac66507 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:38.914340 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c470c-4eae-4449-9b8e-ef2c99a0969a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:38.951971 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384004c-e0e7-4b85-8439-ffd1bcce82b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:38.952583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384004c-e0e7-4b85-8439-ffd1bcce82b7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:38.952864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384004c-e0e7-4b85-8439-ffd1bcce82b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:38.953136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384004c-e0e7-4b85-8439-ffd1bcce82b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:38.953809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384004c-e0e7-4b85-8439-ffd1bcce82b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:38.954113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384004c-e0e7-4b85-8439-ffd1bcce82b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:38.955537 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384004c-e0e7-4b85-8439-ffd1bcce82b7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:38.963912 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 248/493] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:38.964909 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384004c-e0e7-4b85-8439-ffd1bcce82b7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:38.969647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:38.992558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233aae8-5f44-428e-ac93-025fbac66507 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:39.014942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:39.027470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae5b5f-7bf0-49f7-8a29-c9cb9be65380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:39.028097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae5b5f-7bf0-49f7-8a29-c9cb9be65380 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.028371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae5b5f-7bf0-49f7-8a29-c9cb9be65380 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.028639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae5b5f-7bf0-49f7-8a29-c9cb9be65380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.039180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae5b5f-7bf0-49f7-8a29-c9cb9be65380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:39.040662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae5b5f-7bf0-49f7-8a29-c9cb9be65380 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-dbae5b5f-7bf0-49f7-8a29-c9cb9be65380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:39.054632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:39.055595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:39.056557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:39.056557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:39.056557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:39.056557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 246/494] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:39.058657 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b9214-6daa-4f06-9dd8-8b3ac4f08f6f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.058950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b9214-6daa-4f06-9dd8-8b3ac4f08f6f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.059275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b9214-6daa-4f06-9dd8-8b3ac4f08f6f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.061172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 247/495] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:39.062253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ecf9c-979c-4b97-b43c-ca5b56e91dd4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.062617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ecf9c-979c-4b97-b43c-ca5b56e91dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.062875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ecf9c-979c-4b97-b43c-ca5b56e91dd4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.064223 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 248/496] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:39.064805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ecf9c-979c-4b97-b43c-ca5b56e91dd4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:39.070279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384004c-e0e7-4b85-8439-ffd1bcce82b7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:39.104056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d8e0a-fa4e-43bb-8fe3-cd9e1fc0acbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:39.104654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d8e0a-fa4e-43bb-8fe3-cd9e1fc0acbe None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.104935 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d8e0a-fa4e-43bb-8fe3-cd9e1fc0acbe None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.105210 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d8e0a-fa4e-43bb-8fe3-cd9e1fc0acbe None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.105848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d8e0a-fa4e-43bb-8fe3-cd9e1fc0acbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:39.106150 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d8e0a-fa4e-43bb-8fe3-cd9e1fc0acbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:39.108064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d8e0a-fa4e-43bb-8fe3-cd9e1fc0acbe None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:39.119518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 249/497] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:39.120311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d8e0a-fa4e-43bb-8fe3-cd9e1fc0acbe None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:39.166651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ecf9c-979c-4b97-b43c-ca5b56e91dd4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:39.194734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:39.204114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdfc667-cecc-41c2-95e2-f9a271bc709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:39.204748 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfc667-cecc-41c2-95e2-f9a271bc709d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.205029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfc667-cecc-41c2-95e2-f9a271bc709d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.205301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfc667-cecc-41c2-95e2-f9a271bc709d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.215827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdfc667-cecc-41c2-95e2-f9a271bc709d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:39.217324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdfc667-cecc-41c2-95e2-f9a271bc709d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:39.230720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d8e0a-fa4e-43bb-8fe3-cd9e1fc0acbe None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9bdfc667-cecc-41c2-95e2-f9a271bc709d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:39.231153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:39.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:39.233426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:39.233426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:39.233426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:39.233426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 249/498] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:39.237226 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696468c-05eb-4bab-bc7e-52163bc9567d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.237226 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696468c-05eb-4bab-bc7e-52163bc9567d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.237226 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696468c-05eb-4bab-bc7e-52163bc9567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.247550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:39.252857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1696468c-05eb-4bab-bc7e-52163bc9567d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:39.254035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1696468c-05eb-4bab-bc7e-52163bc9567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:39.273073 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197d656-0154-48cc-b11f-4db37666f22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:39.273683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197d656-0154-48cc-b11f-4db37666f22c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.273965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197d656-0154-48cc-b11f-4db37666f22c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.274239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197d656-0154-48cc-b11f-4db37666f22c None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.274876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197d656-0154-48cc-b11f-4db37666f22c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:39.275212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197d656-0154-48cc-b11f-4db37666f22c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:39.276559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197d656-0154-48cc-b11f-4db37666f22c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:39.287684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 250/499] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:39.288841 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f87b3-6b12-4bfe-b845-7c374038e591 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.289129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f87b3-6b12-4bfe-b845-7c374038e591 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.289408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f87b3-6b12-4bfe-b845-7c374038e591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.307282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-224f87b3-6b12-4bfe-b845-7c374038e591 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:39.308461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-224f87b3-6b12-4bfe-b845-7c374038e591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:39.320359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 250/500] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:39.321187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696468c-05eb-4bab-bc7e-52163bc9567d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:39.392699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 251/501] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:39.393666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f87b3-6b12-4bfe-b845-7c374038e591 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:39.437342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696468c-05eb-4bab-bc7e-52163bc9567d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:39.449597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:39.485321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35bbc2-130d-43c3-a567-1219ddf95fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:39.485946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35bbc2-130d-43c3-a567-1219ddf95fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.486221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35bbc2-130d-43c3-a567-1219ddf95fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.486486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35bbc2-130d-43c3-a567-1219ddf95fe2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.497840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35bbc2-130d-43c3-a567-1219ddf95fe2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:39.499232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35bbc2-130d-43c3-a567-1219ddf95fe2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5d35bbc2-130d-43c3-a567-1219ddf95fe2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:39.513573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:39.514477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:39.515403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:39.515403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:39.515403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:39.515403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:39.515403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:39.515403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:39.515403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:39.515403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:39.515403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:48:39.515403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:39.519417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 251/502] 10.222.0.86 () {56 vars in 1093 bytes} [Mon Aug 29 18:48:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:39.520313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35bbc2-130d-43c3-a567-1219ddf95fe2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:39.526775 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:39.533435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f87b3-6b12-4bfe-b845-7c374038e591 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:39.580517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048775c-c458-4a5f-8f47-8f2af4fd8f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:39.581782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048775c-c458-4a5f-8f47-8f2af4fd8f4b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.582480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048775c-c458-4a5f-8f47-8f2af4fd8f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.582972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048775c-c458-4a5f-8f47-8f2af4fd8f4b None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.584153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048775c-c458-4a5f-8f47-8f2af4fd8f4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:39.584648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048775c-c458-4a5f-8f47-8f2af4fd8f4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:39.586816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048775c-c458-4a5f-8f47-8f2af4fd8f4b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:39.601703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 252/503] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:39.602817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048775c-c458-4a5f-8f47-8f2af4fd8f4b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:39.640883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35bbc2-130d-43c3-a567-1219ddf95fe2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:39.674957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e610f7-a7ba-4f76-b362-4fef0f362fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:39.675690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e610f7-a7ba-4f76-b362-4fef0f362fbe None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.675887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e610f7-a7ba-4f76-b362-4fef0f362fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.676164 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e610f7-a7ba-4f76-b362-4fef0f362fbe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.685731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e610f7-a7ba-4f76-b362-4fef0f362fbe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:39.687227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e610f7-a7ba-4f76-b362-4fef0f362fbe None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-22e610f7-a7ba-4f76-b362-4fef0f362fbe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:39.699170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:39.700110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:39.701047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:39.701047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:39.701047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:39.701047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:39.701047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:39.701047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:39.701047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:39.701047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:39.701047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:48:39.701047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:39.701047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 252/504] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:39.701604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e610f7-a7ba-4f76-b362-4fef0f362fbe None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:39.716059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048775c-c458-4a5f-8f47-8f2af4fd8f4b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:39.733194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:39.750833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec620-5262-4eca-882a-ae975287f2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:39.751399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec620-5262-4eca-882a-ae975287f2ed None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.751680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec620-5262-4eca-882a-ae975287f2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.751953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec620-5262-4eca-882a-ae975287f2ed None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.752674 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ec620-5262-4eca-882a-ae975287f2ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:39.752984 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ec620-5262-4eca-882a-ae975287f2ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:39.754692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ec620-5262-4eca-882a-ae975287f2ed None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:39.791225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 253/505] 10.222.0.86 () {56 vars in 1133 bytes} [Mon Aug 29 18:48:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:39.792050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec620-5262-4eca-882a-ae975287f2ed None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:39.805166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e610f7-a7ba-4f76-b362-4fef0f362fbe None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:39.811757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:39.839342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b6990-1699-4e64-ab83-625489aaca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:39.839972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b6990-1699-4e64-ab83-625489aaca6b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.840251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b6990-1699-4e64-ab83-625489aaca6b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.840525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b6990-1699-4e64-ab83-625489aaca6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.851872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b6990-1699-4e64-ab83-625489aaca6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-402b6990-1699-4e64-ab83-625489aaca6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:39.857559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:39.858660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 253/506] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:39.859513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b6990-1699-4e64-ab83-625489aaca6b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:39.894539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec620-5262-4eca-882a-ae975287f2ed None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:39.928071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a855c-5672-4ab8-bde3-769fdaabb9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:39.928692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a855c-5672-4ab8-bde3-769fdaabb9ac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:39.928965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a855c-5672-4ab8-bde3-769fdaabb9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:39.929233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a855c-5672-4ab8-bde3-769fdaabb9ac None admin] PATH_INFO: `/v3/users/glance` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:39.936580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a855c-5672-4ab8-bde3-769fdaabb9ac None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:39.938060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a855c-5672-4ab8-bde3-769fdaabb9ac None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:39.950718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-478a855c-5672-4ab8-bde3-769fdaabb9ac None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:39.950835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:39.951718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:39.952602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:39.952602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:39.952602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:39.952602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:39.952602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:39.952602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:39.952602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:39.952602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:39.952602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:48:39.952602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:39.952602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 254/507] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:39.953625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a855c-5672-4ab8-bde3-769fdaabb9ac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:39.965526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b6990-1699-4e64-ab83-625489aaca6b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:39.999992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e973b92-edbf-42a4-94fb-9f075d811c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:40.000606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e973b92-edbf-42a4-94fb-9f075d811c66 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:40.000951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e973b92-edbf-42a4-94fb-9f075d811c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:40.001163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e973b92-edbf-42a4-94fb-9f075d811c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:40.011428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e973b92-edbf-42a4-94fb-9f075d811c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:40.012866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e973b92-edbf-42a4-94fb-9f075d811c66 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2e973b92-edbf-42a4-94fb-9f075d811c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:40.026689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:40.027811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:40.028668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:40.028668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:40.028668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:40.028668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 254/508] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:40.028668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e973b92-edbf-42a4-94fb-9f075d811c66 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:40.033049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:40.055941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a855c-5672-4ab8-bde3-769fdaabb9ac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:40.089673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6703b-6bc7-40f7-b9fe-54ad8e24d7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:40.090271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6703b-6bc7-40f7-b9fe-54ad8e24d7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:40.090547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6703b-6bc7-40f7-b9fe-54ad8e24d7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:40.090816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6703b-6bc7-40f7-b9fe-54ad8e24d7b3 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:40.091601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6703b-6bc7-40f7-b9fe-54ad8e24d7b3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:40.091901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6703b-6bc7-40f7-b9fe-54ad8e24d7b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:40.093328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6703b-6bc7-40f7-b9fe-54ad8e24d7b3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:40.110611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 255/509] 10.222.0.86 () {56 vars in 1137 bytes} [Mon Aug 29 18:48:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:40.111436 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6703b-6bc7-40f7-b9fe-54ad8e24d7b3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:40.129551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e973b92-edbf-42a4-94fb-9f075d811c66 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:40.164864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1ae85-119f-466b-8218-b6dfa18cb017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:40.165474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ae85-119f-466b-8218-b6dfa18cb017 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:40.165750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ae85-119f-466b-8218-b6dfa18cb017 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:40.166020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ae85-119f-466b-8218-b6dfa18cb017 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:40.178219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:40.209757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6703b-6bc7-40f7-b9fe-54ad8e24d7b3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:40.239646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1ae85-119f-466b-8218-b6dfa18cb017 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:40.241115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1ae85-119f-466b-8218-b6dfa18cb017 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:40.250328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5162bc-e4db-409f-8e17-9d4111389ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:40.250935 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5162bc-e4db-409f-8e17-9d4111389ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:40.251292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5162bc-e4db-409f-8e17-9d4111389ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:40.251519 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5162bc-e4db-409f-8e17-9d4111389ce3 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:40.252178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5162bc-e4db-409f-8e17-9d4111389ce3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:40.252478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5162bc-e4db-409f-8e17-9d4111389ce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:40.253825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5162bc-e4db-409f-8e17-9d4111389ce3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:40.256323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:40.263025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 256/510] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:40.263830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5162bc-e4db-409f-8e17-9d4111389ce3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:40.344186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 255/511] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:40.345024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1ae85-119f-466b-8218-b6dfa18cb017 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:40.370346 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5162bc-e4db-409f-8e17-9d4111389ce3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:40.388427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:40.403256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0478c-88e3-4cbe-b7aa-7c70ebb14be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:40.404396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0478c-88e3-4cbe-b7aa-7c70ebb14be8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:40.404676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0478c-88e3-4cbe-b7aa-7c70ebb14be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:40.404944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0478c-88e3-4cbe-b7aa-7c70ebb14be8 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:40.405663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0478c-88e3-4cbe-b7aa-7c70ebb14be8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:40.405962 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0478c-88e3-4cbe-b7aa-7c70ebb14be8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:40.407409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0478c-88e3-4cbe-b7aa-7c70ebb14be8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:40.425028 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 257/512] 10.222.0.86 () {56 vars in 1137 bytes} [Mon Aug 29 18:48:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:40.425826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0478c-88e3-4cbe-b7aa-7c70ebb14be8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:40.449046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1ae85-119f-466b-8218-b6dfa18cb017 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:40.481630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:40.483794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0186a8-ec43-4a5f-9535-5ad9b3e7c795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:40.484261 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0186a8-ec43-4a5f-9535-5ad9b3e7c795 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:40.484534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0186a8-ec43-4a5f-9535-5ad9b3e7c795 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:40.484800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0186a8-ec43-4a5f-9535-5ad9b3e7c795 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:40.485429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0186a8-ec43-4a5f-9535-5ad9b3e7c795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:40.485724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0186a8-ec43-4a5f-9535-5ad9b3e7c795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:40.487122 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0186a8-ec43-4a5f-9535-5ad9b3e7c795 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:40.497988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 256/513] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:40.498767 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0186a8-ec43-4a5f-9535-5ad9b3e7c795 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:40.533571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0478c-88e3-4cbe-b7aa-7c70ebb14be8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:40.583970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ff79b-e185-4fa5-b528-45aa773a2d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:40.584596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ff79b-e185-4fa5-b528-45aa773a2d5f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:40.584895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ff79b-e185-4fa5-b528-45aa773a2d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:40.585139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ff79b-e185-4fa5-b528-45aa773a2d5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:40.602194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0186a8-ec43-4a5f-9535-5ad9b3e7c795 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:40.603783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:40.641121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a7319-80d1-4f16-99df-43ea22277be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:40.641744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a7319-80d1-4f16-99df-43ea22277be9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:40.642023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a7319-80d1-4f16-99df-43ea22277be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:40.642292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a7319-80d1-4f16-99df-43ea22277be9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:40.652887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a7319-80d1-4f16-99df-43ea22277be9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:40.654988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a7319-80d1-4f16-99df-43ea22277be9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:40.656956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ff79b-e185-4fa5-b528-45aa773a2d5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:40.658376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ff79b-e185-4fa5-b528-45aa773a2d5f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a91a7319-80d1-4f16-99df-43ea22277be9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:40.669256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:40.670171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:40.671031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:40.671031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:40.671031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:40.671031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 257/514] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:40.671258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a7319-80d1-4f16-99df-43ea22277be9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:40.726942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:40.771761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 258/515] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:40.772586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ff79b-e185-4fa5-b528-45aa773a2d5f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:40.789859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a7319-80d1-4f16-99df-43ea22277be9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:40.822920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7538e1-3290-4f3f-9bc3-720a1e603453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:40.823573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7538e1-3290-4f3f-9bc3-720a1e603453 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:40.823853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7538e1-3290-4f3f-9bc3-720a1e603453 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:40.824126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7538e1-3290-4f3f-9bc3-720a1e603453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:40.835696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7538e1-3290-4f3f-9bc3-720a1e603453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:40.837149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7538e1-3290-4f3f-9bc3-720a1e603453 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2a7538e1-3290-4f3f-9bc3-720a1e603453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:40.851892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:40.852850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:40.853684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:40.853684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:40.853684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:40.853684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 258/516] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:40.853684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7538e1-3290-4f3f-9bc3-720a1e603453 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:40.863875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:40.872952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ff79b-e185-4fa5-b528-45aa773a2d5f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:40.908341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199b75a-0c35-4951-9d1e-29cb8e493410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:40.908950 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199b75a-0c35-4951-9d1e-29cb8e493410 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:40.909231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199b75a-0c35-4951-9d1e-29cb8e493410 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:40.909503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199b75a-0c35-4951-9d1e-29cb8e493410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:40.933913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:40.965750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7538e1-3290-4f3f-9bc3-720a1e603453 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:40.978915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199b75a-0c35-4951-9d1e-29cb8e493410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:40.980414 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199b75a-0c35-4951-9d1e-29cb8e493410 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.003051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd0a43-1967-4036-84be-b9da30e4f7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:41.003666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd0a43-1967-4036-84be-b9da30e4f7ce None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:41.003946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd0a43-1967-4036-84be-b9da30e4f7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:41.004223 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd0a43-1967-4036-84be-b9da30e4f7ce None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:41.004865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd0a43-1967-4036-84be-b9da30e4f7ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:41.006286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd0a43-1967-4036-84be-b9da30e4f7ce None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.027723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 259/517] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:48:40 2022] POST /identity/v3/users => generated 247 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:48:41.028548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd0a43-1967-4036-84be-b9da30e4f7ce None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:41.089673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 259/518] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:41.090863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199b75a-0c35-4951-9d1e-29cb8e493410 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:41.118797 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:41.157636 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:41.166421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd0a43-1967-4036-84be-b9da30e4f7ce None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:41.206650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232ab78-db5c-481a-8eb7-b49607b74806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:41.207293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232ab78-db5c-481a-8eb7-b49607b74806 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:41.207569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232ab78-db5c-481a-8eb7-b49607b74806 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:41.207838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232ab78-db5c-481a-8eb7-b49607b74806 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:41.208480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232ab78-db5c-481a-8eb7-b49607b74806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:41.208778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232ab78-db5c-481a-8eb7-b49607b74806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:41.210075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199b75a-0c35-4951-9d1e-29cb8e493410 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:41.210195 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232ab78-db5c-481a-8eb7-b49607b74806 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.223361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 260/519] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:41.224195 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232ab78-db5c-481a-8eb7-b49607b74806 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:41.251653 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658209f0-17eb-4f48-87ea-92af3752c7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:41.252290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658209f0-17eb-4f48-87ea-92af3752c7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:41.252573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658209f0-17eb-4f48-87ea-92af3752c7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:41.252849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658209f0-17eb-4f48-87ea-92af3752c7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:41.264013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658209f0-17eb-4f48-87ea-92af3752c7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:41.265543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658209f0-17eb-4f48-87ea-92af3752c7e0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-658209f0-17eb-4f48-87ea-92af3752c7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:41.280952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:41.281925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:41.282786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:41.282786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:41.282786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:41.282786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 260/520] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:41.283871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658209f0-17eb-4f48-87ea-92af3752c7e0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:41.342348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232ab78-db5c-481a-8eb7-b49607b74806 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:41.365968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:41.379245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51938e60-a31a-4553-8c72-f7d0909720f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:41.379857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938e60-a31a-4553-8c72-f7d0909720f7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:41.380136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938e60-a31a-4553-8c72-f7d0909720f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:41.380414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938e60-a31a-4553-8c72-f7d0909720f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:41.381045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51938e60-a31a-4553-8c72-f7d0909720f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:41.381348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51938e60-a31a-4553-8c72-f7d0909720f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:41.382714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51938e60-a31a-4553-8c72-f7d0909720f7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.392739 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658209f0-17eb-4f48-87ea-92af3752c7e0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:41.399603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 261/521] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:41.400409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51938e60-a31a-4553-8c72-f7d0909720f7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:41.402460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:41.431820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76061c21-17aa-4e07-b7da-506cbfdc8059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:41.432474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061c21-17aa-4e07-b7da-506cbfdc8059 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:41.432761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061c21-17aa-4e07-b7da-506cbfdc8059 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:41.433178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061c21-17aa-4e07-b7da-506cbfdc8059 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:41.445187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76061c21-17aa-4e07-b7da-506cbfdc8059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:41.446640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76061c21-17aa-4e07-b7da-506cbfdc8059 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-76061c21-17aa-4e07-b7da-506cbfdc8059 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:41.461463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:41.462598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:41.463457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:41.463457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:41.463457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:41.463457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 261/522] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:41.463457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76061c21-17aa-4e07-b7da-506cbfdc8059 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:41.517914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51938e60-a31a-4553-8c72-f7d0909720f7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:41.553062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306e6ba-a290-4a09-9b48-d57f167a81e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:41.553696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306e6ba-a290-4a09-9b48-d57f167a81e4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:41.554380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306e6ba-a290-4a09-9b48-d57f167a81e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:41.554380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306e6ba-a290-4a09-9b48-d57f167a81e4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:41.563277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306e6ba-a290-4a09-9b48-d57f167a81e4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:41.564951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306e6ba-a290-4a09-9b48-d57f167a81e4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6306e6ba-a290-4a09-9b48-d57f167a81e4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:41.577371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:41.578440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:41.579470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:41.579470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:41.579470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:41.579470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:41.579470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:41.579470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:41.579470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:41.579470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:41.579470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:48:41.579470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:41.581495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76061c21-17aa-4e07-b7da-506cbfdc8059 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:41.583241 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 262/523] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:41.584080 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306e6ba-a290-4a09-9b48-d57f167a81e4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:41.618758 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff1560-4aae-4ea0-845d-ba8f31518af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:41.619315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff1560-4aae-4ea0-845d-ba8f31518af8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:41.619596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff1560-4aae-4ea0-845d-ba8f31518af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:41.619874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff1560-4aae-4ea0-845d-ba8f31518af8 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:41.620509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff1560-4aae-4ea0-845d-ba8f31518af8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:41.620810 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff1560-4aae-4ea0-845d-ba8f31518af8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:41.622165 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff1560-4aae-4ea0-845d-ba8f31518af8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.627145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:41.635466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 262/524] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:41.636334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff1560-4aae-4ea0-845d-ba8f31518af8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:41.661492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:41.699638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306e6ba-a290-4a09-9b48-d57f167a81e4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:41.735297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02687a-0162-44ae-b41a-9538fca141c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:41.735928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02687a-0162-44ae-b41a-9538fca141c2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:41.736206 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02687a-0162-44ae-b41a-9538fca141c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:41.736478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02687a-0162-44ae-b41a-9538fca141c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:41.743136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02687a-0162-44ae-b41a-9538fca141c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:41.744584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02687a-0162-44ae-b41a-9538fca141c2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.752220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff1560-4aae-4ea0-845d-ba8f31518af8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5c02687a-0162-44ae-b41a-9538fca141c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:41.756537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:41.757359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:41.758328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:41.758328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:41.758328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:41.758328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:41.758328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:41.758328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:41.758328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:41.758328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:41.758328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 263/525] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:41.758328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02687a-0162-44ae-b41a-9538fca141c2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:41.789346 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b93f6-bde7-4f9c-b3f4-5aadb5e0bceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:41.790582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b93f6-bde7-4f9c-b3f4-5aadb5e0bceb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:41.791083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b93f6-bde7-4f9c-b3f4-5aadb5e0bceb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:41.791527 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b93f6-bde7-4f9c-b3f4-5aadb5e0bceb None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:41.792340 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b93f6-bde7-4f9c-b3f4-5aadb5e0bceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:41.792844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b93f6-bde7-4f9c-b3f4-5aadb5e0bceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:41.794503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b93f6-bde7-4f9c-b3f4-5aadb5e0bceb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.806922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 263/526] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:41.807989 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b93f6-bde7-4f9c-b3f4-5aadb5e0bceb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:41.870245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02687a-0162-44ae-b41a-9538fca141c2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:41.871754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:41.901251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:41.907522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aef5d8-08a3-42e6-a6a7-28c3e48157df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:41.908132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aef5d8-08a3-42e6-a6a7-28c3e48157df None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:41.908414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aef5d8-08a3-42e6-a6a7-28c3e48157df None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:41.908686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aef5d8-08a3-42e6-a6a7-28c3e48157df None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:41.909420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aef5d8-08a3-42e6-a6a7-28c3e48157df None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:41.909723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aef5d8-08a3-42e6-a6a7-28c3e48157df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:41.911170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aef5d8-08a3-42e6-a6a7-28c3e48157df None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.924429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b93f6-bde7-4f9c-b3f4-5aadb5e0bceb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:41.953070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 264/527] 10.222.0.86 () {56 vars in 1139 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:41.953070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aef5d8-08a3-42e6-a6a7-28c3e48157df None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:41.965073 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090305cd-a1d7-4cf7-be4f-206bcebf89c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:41.965791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090305cd-a1d7-4cf7-be4f-206bcebf89c3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:41.966056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090305cd-a1d7-4cf7-be4f-206bcebf89c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:41.966838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090305cd-a1d7-4cf7-be4f-206bcebf89c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:41.973976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090305cd-a1d7-4cf7-be4f-206bcebf89c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:41.975441 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090305cd-a1d7-4cf7-be4f-206bcebf89c3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-090305cd-a1d7-4cf7-be4f-206bcebf89c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:41.987870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:41.988814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:41.989667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:41.989667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:41.989667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:41.989667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:41.989667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:41.989667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:41.989667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:41.989667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:41.989667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 264/528] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:41.990145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090305cd-a1d7-4cf7-be4f-206bcebf89c3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:42.069530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aef5d8-08a3-42e6-a6a7-28c3e48157df None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:42.105293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090305cd-a1d7-4cf7-be4f-206bcebf89c3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:42.106493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a53f3-2a46-4d01-8822-b151db5da487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:42.107225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a53f3-2a46-4d01-8822-b151db5da487 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:42.108250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a53f3-2a46-4d01-8822-b151db5da487 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:42.108250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a53f3-2a46-4d01-8822-b151db5da487 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:42.108537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a53f3-2a46-4d01-8822-b151db5da487 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:42.108777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a53f3-2a46-4d01-8822-b151db5da487 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:42.110430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a53f3-2a46-4d01-8822-b151db5da487 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:42.129348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 265/529] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:42.129935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a53f3-2a46-4d01-8822-b151db5da487 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:42.146795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964250cc-69ba-42bc-bccd-944c291172b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:42.147492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964250cc-69ba-42bc-bccd-944c291172b9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:42.147771 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964250cc-69ba-42bc-bccd-944c291172b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:42.148046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964250cc-69ba-42bc-bccd-944c291172b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:42.155712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964250cc-69ba-42bc-bccd-944c291172b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:42.156707 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:42.157161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964250cc-69ba-42bc-bccd-944c291172b9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:42.160469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-964250cc-69ba-42bc-bccd-944c291172b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:42.170255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:42.171510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:42.172350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:42.172350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:42.172350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:42.172350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:42.172350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:42.172350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:42.172350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:42.172350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:42.178715 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 265/530] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:42.179823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964250cc-69ba-42bc-bccd-944c291172b9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:42.243757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a53f3-2a46-4d01-8822-b151db5da487 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:42.280940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da766ec1-0b9d-4c25-acc8-a67b9d6653b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:42.281546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da766ec1-0b9d-4c25-acc8-a67b9d6653b6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:42.281829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da766ec1-0b9d-4c25-acc8-a67b9d6653b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:42.282104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da766ec1-0b9d-4c25-acc8-a67b9d6653b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:42.289429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964250cc-69ba-42bc-bccd-944c291172b9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:42.323421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad325b2f-bdb6-448c-aa7e-1c0b40723d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:42.324023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad325b2f-bdb6-448c-aa7e-1c0b40723d64 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:42.324307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad325b2f-bdb6-448c-aa7e-1c0b40723d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:42.324581 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad325b2f-bdb6-448c-aa7e-1c0b40723d64 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:42.325263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad325b2f-bdb6-448c-aa7e-1c0b40723d64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:42.326082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad325b2f-bdb6-448c-aa7e-1c0b40723d64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:42.327522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad325b2f-bdb6-448c-aa7e-1c0b40723d64 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:42.350020 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 266/531] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:42.350020 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad325b2f-bdb6-448c-aa7e-1c0b40723d64 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:42.357335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da766ec1-0b9d-4c25-acc8-a67b9d6653b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:42.358835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da766ec1-0b9d-4c25-acc8-a67b9d6653b6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:42.381987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:42.412520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:42.459414 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad325b2f-bdb6-448c-aa7e-1c0b40723d64 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:42.467880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 266/532] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:42.467880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da766ec1-0b9d-4c25-acc8-a67b9d6653b6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:42.498434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a2d5c-af0d-4fe9-9bb8-b68523856a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:42.499062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a2d5c-af0d-4fe9-9bb8-b68523856a06 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:42.499347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a2d5c-af0d-4fe9-9bb8-b68523856a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:42.499640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a2d5c-af0d-4fe9-9bb8-b68523856a06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:42.500451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a2d5c-af0d-4fe9-9bb8-b68523856a06 None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=045b22d046c549f2b53d2ed73b94f936) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:42.500940 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a2d5c-af0d-4fe9-9bb8-b68523856a06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:42.502383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a2d5c-af0d-4fe9-9bb8-b68523856a06 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:42.508101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 267/533] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:42 2022] GET /identity/v3/role_assignments?user.id=045b22d046c549f2b53d2ed73b94f936&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:42.508101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a2d5c-af0d-4fe9-9bb8-b68523856a06 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:42.596872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da766ec1-0b9d-4c25-acc8-a67b9d6653b6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:42.628175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:42.646327 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad2059-a8be-45cd-906d-eb09bd02e5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:42.647023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad2059-a8be-45cd-906d-eb09bd02e5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:42.647422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad2059-a8be-45cd-906d-eb09bd02e5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:42.647788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad2059-a8be-45cd-906d-eb09bd02e5d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:42.648593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad2059-a8be-45cd-906d-eb09bd02e5d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:42.649010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad2059-a8be-45cd-906d-eb09bd02e5d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:42.650561 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad2059-a8be-45cd-906d-eb09bd02e5d3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:42.655040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a2d5c-af0d-4fe9-9bb8-b68523856a06 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:42.665478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 267/534] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:42.666521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad2059-a8be-45cd-906d-eb09bd02e5d3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:42.689655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:42.705216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162030f-4faa-47cb-a083-48ffd0ade44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:42.706440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162030f-4faa-47cb-a083-48ffd0ade44e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:42.707231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162030f-4faa-47cb-a083-48ffd0ade44e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:42.707733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162030f-4faa-47cb-a083-48ffd0ade44e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:42.709280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162030f-4faa-47cb-a083-48ffd0ade44e None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=a3030e2c49614aee892c7693da6b73bf) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:42.709761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162030f-4faa-47cb-a083-48ffd0ade44e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:42.712343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162030f-4faa-47cb-a083-48ffd0ade44e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:42.727676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 268/535] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:42 2022] GET /identity/v3/role_assignments?user.id=a3030e2c49614aee892c7693da6b73bf&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:42.728182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162030f-4faa-47cb-a083-48ffd0ade44e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:42.857862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad2059-a8be-45cd-906d-eb09bd02e5d3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:42.895540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7ff0c-1483-445b-bdd3-8bed1c50bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:42.897024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7ff0c-1483-445b-bdd3-8bed1c50bcdf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:42.897517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7ff0c-1483-445b-bdd3-8bed1c50bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:42.897984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7ff0c-1483-445b-bdd3-8bed1c50bcdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:42.907533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162030f-4faa-47cb-a083-48ffd0ade44e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:42.910780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7ff0c-1483-445b-bdd3-8bed1c50bcdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:42.912487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7ff0c-1483-445b-bdd3-8bed1c50bcdf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-36b7ff0c-1483-445b-bdd3-8bed1c50bcdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:42.928733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:42.929682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:42.930518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:42.930518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:42.930518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:42.932624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 268/536] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:42.944631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7ff0c-1483-445b-bdd3-8bed1c50bcdf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:42.961796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602af5e-4663-4cc8-8716-b5ef270ef4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:42.962396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602af5e-4663-4cc8-8716-b5ef270ef4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:42.962678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602af5e-4663-4cc8-8716-b5ef270ef4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:42.962952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602af5e-4663-4cc8-8716-b5ef270ef4e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:42.963763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602af5e-4663-4cc8-8716-b5ef270ef4e0 None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=9af69888b04c45b09e34d9ff7a0b564f) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:42.964075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602af5e-4663-4cc8-8716-b5ef270ef4e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:42.965607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602af5e-4663-4cc8-8716-b5ef270ef4e0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:42.973968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 269/537] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:42 2022] GET /identity/v3/role_assignments?user.id=9af69888b04c45b09e34d9ff7a0b564f&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:42.979881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:43.005830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afd95-e8d4-4bdf-8e4a-4da33b983fbd None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:43.006169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afd95-e8d4-4bdf-8e4a-4da33b983fbd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:43.006390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afd95-e8d4-4bdf-8e4a-4da33b983fbd None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:43.007985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 270/538] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:48:43.014949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9bc92-9236-4800-b46a-6f6ae5dfdc2f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:43.016721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9bc92-9236-4800-b46a-6f6ae5dfdc2f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:43.016995 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9bc92-9236-4800-b46a-6f6ae5dfdc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:43.062520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d2d9bc92-9236-4800-b46a-6f6ae5dfdc2f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:43.066833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d2d9bc92-9236-4800-b46a-6f6ae5dfdc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:43.138467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7ff0c-1483-445b-bdd3-8bed1c50bcdf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:43.180037 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34659c-2fea-4bac-858a-05e419c14c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:43.180565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34659c-2fea-4bac-858a-05e419c14c00 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:43.180843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34659c-2fea-4bac-858a-05e419c14c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:43.181112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34659c-2fea-4bac-858a-05e419c14c00 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:43.181760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34659c-2fea-4bac-858a-05e419c14c00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:43.182059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34659c-2fea-4bac-858a-05e419c14c00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:43.185646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34659c-2fea-4bac-858a-05e419c14c00 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:43.187602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:43.200119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 269/539] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:43.205442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34659c-2fea-4bac-858a-05e419c14c00 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:43.240701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 271/540] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:43.310082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:43.324580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34659c-2fea-4bac-858a-05e419c14c00 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:43.372325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88670d30-f71d-45e9-8858-4e6fdeb84396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:43.373016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88670d30-f71d-45e9-8858-4e6fdeb84396 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:43.373266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88670d30-f71d-45e9-8858-4e6fdeb84396 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:43.373772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88670d30-f71d-45e9-8858-4e6fdeb84396 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:43.384497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88670d30-f71d-45e9-8858-4e6fdeb84396 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:43.386018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88670d30-f71d-45e9-8858-4e6fdeb84396 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-88670d30-f71d-45e9-8858-4e6fdeb84396 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:43.400092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:43.400996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:43.401856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:43.401856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:43.401856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:43.401856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:43.401856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:43.401856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:43.401856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:43.401856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:43.406456 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 270/541] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:43.408122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9bc92-9236-4800-b46a-6f6ae5dfdc2f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:43.555621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88670d30-f71d-45e9-8858-4e6fdeb84396 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:43.560277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9bc92-9236-4800-b46a-6f6ae5dfdc2f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:43.601931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb36ea1-c46d-42f2-8075-35f8a89eaa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:43.602547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb36ea1-c46d-42f2-8075-35f8a89eaa07 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:43.602832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb36ea1-c46d-42f2-8075-35f8a89eaa07 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:43.603140 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb36ea1-c46d-42f2-8075-35f8a89eaa07 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:43.603879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb36ea1-c46d-42f2-8075-35f8a89eaa07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:43.604195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb36ea1-c46d-42f2-8075-35f8a89eaa07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:43.605674 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb36ea1-c46d-42f2-8075-35f8a89eaa07 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:43.619695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 272/542] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:43.632008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb36ea1-c46d-42f2-8075-35f8a89eaa07 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:43.727386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:43.767471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88670d30-f71d-45e9-8858-4e6fdeb84396 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:43.809316 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb36ea1-c46d-42f2-8075-35f8a89eaa07 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:43.839705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9ad92-5307-4441-bda1-8e57c63c750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:43.841807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ad92-5307-4441-bda1-8e57c63c750a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:43.842827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ad92-5307-4441-bda1-8e57c63c750a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:43.843365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ad92-5307-4441-bda1-8e57c63c750a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:43.856606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f97c5-503a-465b-9929-6957e8dab97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:43.857396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f97c5-503a-465b-9929-6957e8dab97f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:43.858684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f97c5-503a-465b-9929-6957e8dab97f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:43.859145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f97c5-503a-465b-9929-6957e8dab97f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:43.860253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f97c5-503a-465b-9929-6957e8dab97f None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=5ec64944b34d4ef1bd666bb1b7e216da) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:43.861922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f97c5-503a-465b-9929-6957e8dab97f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:43.864726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f97c5-503a-465b-9929-6957e8dab97f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:43.865634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9ad92-5307-4441-bda1-8e57c63c750a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c9b9ad92-5307-4441-bda1-8e57c63c750a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:43.871740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:43.872798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:43.873782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:43.876250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 271/543] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:43.880743 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 273/544] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:43 2022] GET /identity/v3/role_assignments?user.id=5ec64944b34d4ef1bd666bb1b7e216da&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:43.882866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9ad92-5307-4441-bda1-8e57c63c750a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:43.901503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:44.000851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9ad92-5307-4441-bda1-8e57c63c750a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:44.040616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d97794-9b7a-4f13-94c6-f88db1c69e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:44.041464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d97794-9b7a-4f13-94c6-f88db1c69e8e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:44.041882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d97794-9b7a-4f13-94c6-f88db1c69e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:44.042349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d97794-9b7a-4f13-94c6-f88db1c69e8e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:44.043182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d97794-9b7a-4f13-94c6-f88db1c69e8e None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:44.043661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d97794-9b7a-4f13-94c6-f88db1c69e8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:44.045477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d97794-9b7a-4f13-94c6-f88db1c69e8e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:44.055592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 272/545] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:44.064211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f97c5-503a-465b-9929-6957e8dab97f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:44.194740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f97c5-503a-465b-9929-6957e8dab97f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:44.203556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:44.231450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74692655-590e-4d4a-ac21-6e58842bf215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:44.232108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74692655-590e-4d4a-ac21-6e58842bf215 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:44.232386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74692655-590e-4d4a-ac21-6e58842bf215 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:44.232656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74692655-590e-4d4a-ac21-6e58842bf215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:44.243611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74692655-590e-4d4a-ac21-6e58842bf215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:44.245104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74692655-590e-4d4a-ac21-6e58842bf215 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-74692655-590e-4d4a-ac21-6e58842bf215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:44.260681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:44.261638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:44.263708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:44.263708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:44.263708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:44.263708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 274/546] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:44.269006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d97794-9b7a-4f13-94c6-f88db1c69e8e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:44.350751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:44.379644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d97794-9b7a-4f13-94c6-f88db1c69e8e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:44.414987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190c9e1-f7d3-4c4a-8bda-8787345bb440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:44.415618 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190c9e1-f7d3-4c4a-8bda-8787345bb440 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:44.415948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190c9e1-f7d3-4c4a-8bda-8787345bb440 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:44.416219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190c9e1-f7d3-4c4a-8bda-8787345bb440 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:44.416854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190c9e1-f7d3-4c4a-8bda-8787345bb440 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:44.417154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190c9e1-f7d3-4c4a-8bda-8787345bb440 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:44.418570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190c9e1-f7d3-4c4a-8bda-8787345bb440 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:44.431898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 273/547] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:44.438134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74692655-590e-4d4a-ac21-6e58842bf215 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:44.569946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74692655-590e-4d4a-ac21-6e58842bf215 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:44.619953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b32566-d2df-4d18-a15e-e46ce3f82c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:44.620504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32566-d2df-4d18-a15e-e46ce3f82c40 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:44.620778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32566-d2df-4d18-a15e-e46ce3f82c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:44.621045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32566-d2df-4d18-a15e-e46ce3f82c40 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:44.633427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b32566-d2df-4d18-a15e-e46ce3f82c40 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:44.634864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b32566-d2df-4d18-a15e-e46ce3f82c40 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-65b32566-d2df-4d18-a15e-e46ce3f82c40 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:44.649849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:44.652142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:44.655590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:44.655590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:44.655590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:44.655590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:44.655590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:44.655590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:44.655590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:44.655590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:44.655590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:48:44.655590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:44.655590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 275/548] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:48:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:44.659062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190c9e1-f7d3-4c4a-8bda-8787345bb440 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:44.788717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190c9e1-f7d3-4c4a-8bda-8787345bb440 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:44.816530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48139de-eb88-4765-bf3a-743cae48a0cc None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:44.816855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48139de-eb88-4765-bf3a-743cae48a0cc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:44.817069 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48139de-eb88-4765-bf3a-743cae48a0cc None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:44.818548 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 276/549] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:44.824855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e38f4-7033-429e-8921-d9dfc1b34ed3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:44.824855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e38f4-7033-429e-8921-d9dfc1b34ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:44.824855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e38f4-7033-429e-8921-d9dfc1b34ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:44.830460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6e382-ab7b-490b-8b99-feac43b48150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:44.831092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6e382-ab7b-490b-8b99-feac43b48150 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:44.831373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6e382-ab7b-490b-8b99-feac43b48150 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:44.831640 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6e382-ab7b-490b-8b99-feac43b48150 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:44.832367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6e382-ab7b-490b-8b99-feac43b48150 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:44.832666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6e382-ab7b-490b-8b99-feac43b48150 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:44.834081 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6e382-ab7b-490b-8b99-feac43b48150 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:44.839039 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:44.855172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 274/550] 10.222.0.86 () {56 vars in 1135 bytes} [Mon Aug 29 18:48:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:44.861748 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-eb5e38f4-7033-429e-8921-d9dfc1b34ed3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:44.861913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6e382-ab7b-490b-8b99-feac43b48150 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:44.862955 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eb5e38f4-7033-429e-8921-d9dfc1b34ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:44.879506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:44.942427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 277/551] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:45.006248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6e382-ab7b-490b-8b99-feac43b48150 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:45.043214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a7804-d36b-4e50-a662-316a4145bd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:45.043821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a7804-d36b-4e50-a662-316a4145bd2a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:45.044096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a7804-d36b-4e50-a662-316a4145bd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:45.044452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a7804-d36b-4e50-a662-316a4145bd2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:45.105868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:45.115979 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a7804-d36b-4e50-a662-316a4145bd2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:45.117692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a7804-d36b-4e50-a662-316a4145bd2a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:45.220016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 275/552] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:48:45.225496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e38f4-7033-429e-8921-d9dfc1b34ed3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:45.293142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a7804-d36b-4e50-a662-316a4145bd2a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:45.371364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e38f4-7033-429e-8921-d9dfc1b34ed3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:45.401909 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:45.411137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e901d2-4640-4d33-9f7a-a4b1731dc878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:45.411903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e901d2-4640-4d33-9f7a-a4b1731dc878 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:45.412189 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e901d2-4640-4d33-9f7a-a4b1731dc878 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:45.412459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e901d2-4640-4d33-9f7a-a4b1731dc878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:45.415952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a7804-d36b-4e50-a662-316a4145bd2a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:45.423713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e901d2-4640-4d33-9f7a-a4b1731dc878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:45.425179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e901d2-4640-4d33-9f7a-a4b1731dc878 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:45.435139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e8e901d2-4640-4d33-9f7a-a4b1731dc878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:45.442986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:45.444263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:45.445153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:45.445153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:45.445153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:45.446556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 278/553] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:45.447891 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7955c0-7a5b-4c25-b7a8-057fbc44a415 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:45.448301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7955c0-7a5b-4c25-b7a8-057fbc44a415 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:45.448692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7955c0-7a5b-4c25-b7a8-057fbc44a415 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:45.450373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 279/554] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:45.452260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7112-7bf8-4e1f-b92a-fc5c46a67104 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:45.452545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7112-7bf8-4e1f-b92a-fc5c46a67104 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:45.452813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7112-7bf8-4e1f-b92a-fc5c46a67104 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:45.454088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 280/555] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:45.455470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7112-7bf8-4e1f-b92a-fc5c46a67104 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:45.462779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882ca52-16bf-49e4-b64a-e4f709f500e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:45.463442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882ca52-16bf-49e4-b64a-e4f709f500e3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:45.463723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882ca52-16bf-49e4-b64a-e4f709f500e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:45.463996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882ca52-16bf-49e4-b64a-e4f709f500e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:45.475989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882ca52-16bf-49e4-b64a-e4f709f500e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7882ca52-16bf-49e4-b64a-e4f709f500e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:45.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:45.481976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:45.482910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 276/556] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:45.485034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027490a-0450-42cf-abe9-09a36f357c8a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:45.485034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027490a-0450-42cf-abe9-09a36f357c8a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:45.485034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027490a-0450-42cf-abe9-09a36f357c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:45.504013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7027490a-0450-42cf-abe9-09a36f357c8a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:45.505216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7027490a-0450-42cf-abe9-09a36f357c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:45.570279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7112-7bf8-4e1f-b92a-fc5c46a67104 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:45.573525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 277/557] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:45.580416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35557b48-85d3-4431-90a5-8795045ebe2c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:45.581111 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35557b48-85d3-4431-90a5-8795045ebe2c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:45.581391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35557b48-85d3-4431-90a5-8795045ebe2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:45.605853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cd993-0944-420a-a902-4670d8264eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:45.606464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-35557b48-85d3-4431-90a5-8795045ebe2c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:45.607057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cd993-0944-420a-a902-4670d8264eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:45.607530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cd993-0944-420a-a902-4670d8264eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:45.607803 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-35557b48-85d3-4431-90a5-8795045ebe2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:45.608209 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cd993-0944-420a-a902-4670d8264eb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:45.609049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cd993-0944-420a-a902-4670d8264eb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:45.609539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cd993-0944-420a-a902-4670d8264eb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:45.611106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cd993-0944-420a-a902-4670d8264eb5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:45.627332 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 281/558] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:45.628408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cd993-0944-420a-a902-4670d8264eb5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:45.703867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:45.714448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 278/559] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:45.714448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35557b48-85d3-4431-90a5-8795045ebe2c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:45.740515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:45.784524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cd993-0944-420a-a902-4670d8264eb5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:45.850730 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70573833-068a-4510-aa1d-9e21c40f13bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:45.851380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70573833-068a-4510-aa1d-9e21c40f13bc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:45.851658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70573833-068a-4510-aa1d-9e21c40f13bc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:45.851930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70573833-068a-4510-aa1d-9e21c40f13bc None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:45.852596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70573833-068a-4510-aa1d-9e21c40f13bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:45.852898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70573833-068a-4510-aa1d-9e21c40f13bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:45.854254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70573833-068a-4510-aa1d-9e21c40f13bc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:45.863371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 282/560] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:45.864192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70573833-068a-4510-aa1d-9e21c40f13bc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:45.875990 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35557b48-85d3-4431-90a5-8795045ebe2c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:45.914220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67657bd5-bf36-47c5-8b72-fe75afa7c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:45.915445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67657bd5-bf36-47c5-8b72-fe75afa7c4ac None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:45.915920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67657bd5-bf36-47c5-8b72-fe75afa7c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:45.916686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67657bd5-bf36-47c5-8b72-fe75afa7c4ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:45.924840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67657bd5-bf36-47c5-8b72-fe75afa7c4ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:45.926711 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67657bd5-bf36-47c5-8b72-fe75afa7c4ac None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-67657bd5-bf36-47c5-8b72-fe75afa7c4ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:45.939708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:45.940609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:45.941470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:45.941470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:45.941470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:45.941470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:45.941470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:45.941470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:45.941470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:45.941470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:45.944207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 279/561] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:45.945279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67657bd5-bf36-47c5-8b72-fe75afa7c4ac None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:45.967287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70573833-068a-4510-aa1d-9e21c40f13bc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:45.990496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:46.003920 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a2dea-b03f-4fe1-b0fd-4bb03fdb402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.004535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2dea-b03f-4fe1-b0fd-4bb03fdb402a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.004814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2dea-b03f-4fe1-b0fd-4bb03fdb402a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.005084 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2dea-b03f-4fe1-b0fd-4bb03fdb402a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.016260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a2dea-b03f-4fe1-b0fd-4bb03fdb402a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-036a2dea-b03f-4fe1-b0fd-4bb03fdb402a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:46.021982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:46.022911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:46.023776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 283/562] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:46.024461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a2dea-b03f-4fe1-b0fd-4bb03fdb402a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:46.030874 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:46.069548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67657bd5-bf36-47c5-8b72-fe75afa7c4ac None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:46.104976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542ffce-9969-4540-bf3a-2578dc6f4a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.105616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542ffce-9969-4540-bf3a-2578dc6f4a1d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.105895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542ffce-9969-4540-bf3a-2578dc6f4a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.106165 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542ffce-9969-4540-bf3a-2578dc6f4a1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.117804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542ffce-9969-4540-bf3a-2578dc6f4a1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:46.119272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542ffce-9969-4540-bf3a-2578dc6f4a1d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3542ffce-9969-4540-bf3a-2578dc6f4a1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:46.135041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:46.136068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:46.136935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:46.136935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:46.136935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:46.136935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 280/563] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:46.137163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542ffce-9969-4540-bf3a-2578dc6f4a1d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:46.148664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a2dea-b03f-4fe1-b0fd-4bb03fdb402a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:46.183278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a353022-086a-4c76-b165-b646a83589f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.183878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a353022-086a-4c76-b165-b646a83589f8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.184156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a353022-086a-4c76-b165-b646a83589f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.184416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a353022-086a-4c76-b165-b646a83589f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.185816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a353022-086a-4c76-b165-b646a83589f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:46.186126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a353022-086a-4c76-b165-b646a83589f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:46.187575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a353022-086a-4c76-b165-b646a83589f8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:46.199631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 284/564] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:46.200417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a353022-086a-4c76-b165-b646a83589f8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:46.247566 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542ffce-9969-4540-bf3a-2578dc6f4a1d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:46.248598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:46.274485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:46.286697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1726bb-053a-4855-aa26-20c70eac7444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.287255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1726bb-053a-4855-aa26-20c70eac7444 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.287533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1726bb-053a-4855-aa26-20c70eac7444 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.287801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1726bb-053a-4855-aa26-20c70eac7444 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.288454 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1726bb-053a-4855-aa26-20c70eac7444 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:46.288754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1726bb-053a-4855-aa26-20c70eac7444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:46.290115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1726bb-053a-4855-aa26-20c70eac7444 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:46.304256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 281/565] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:46.304256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1726bb-053a-4855-aa26-20c70eac7444 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:46.313060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a353022-086a-4c76-b165-b646a83589f8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:46.348134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2291417-819e-4f41-a20b-ea4abc3542e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.348762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2291417-819e-4f41-a20b-ea4abc3542e3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.349041 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2291417-819e-4f41-a20b-ea4abc3542e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.349314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2291417-819e-4f41-a20b-ea4abc3542e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.359618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2291417-819e-4f41-a20b-ea4abc3542e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-d2291417-819e-4f41-a20b-ea4abc3542e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:46.364373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:46.365396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 285/566] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:46.366254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2291417-819e-4f41-a20b-ea4abc3542e3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:46.402955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1726bb-053a-4855-aa26-20c70eac7444 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:46.435887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b46ae-4103-40fa-954f-202b8fed9e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.436480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b46ae-4103-40fa-954f-202b8fed9e1e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.436754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b46ae-4103-40fa-954f-202b8fed9e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.437020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b46ae-4103-40fa-954f-202b8fed9e1e None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.437738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b46ae-4103-40fa-954f-202b8fed9e1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:46.438033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b46ae-4103-40fa-954f-202b8fed9e1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:46.439491 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b46ae-4103-40fa-954f-202b8fed9e1e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:46.450946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 282/567] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:46.451739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b46ae-4103-40fa-954f-202b8fed9e1e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:46.465847 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2291417-819e-4f41-a20b-ea4abc3542e3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:46.468038 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:46.497519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:46.502804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00e2be-8663-4e6c-b280-fb0b93fc60bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.503435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00e2be-8663-4e6c-b280-fb0b93fc60bf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.503714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00e2be-8663-4e6c-b280-fb0b93fc60bf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.503986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00e2be-8663-4e6c-b280-fb0b93fc60bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.504705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00e2be-8663-4e6c-b280-fb0b93fc60bf None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=dd1b7173843b4da48e69da8b36d59226) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:46.505012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00e2be-8663-4e6c-b280-fb0b93fc60bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:46.506444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00e2be-8663-4e6c-b280-fb0b93fc60bf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:46.511754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 286/568] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:46 2022] GET /identity/v3/role_assignments?user.id=dd1b7173843b4da48e69da8b36d59226&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:46.512978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9482cb5-3383-4a85-b82e-623202c71b7d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.513274 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9482cb5-3383-4a85-b82e-623202c71b7d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.513540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9482cb5-3383-4a85-b82e-623202c71b7d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.514735 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 287/569] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:46.519972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9482cb5-3383-4a85-b82e-623202c71b7d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:46.558678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b46ae-4103-40fa-954f-202b8fed9e1e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:46.596544 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692bee1-dc50-47c3-8046-bda52b7fb672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.597184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692bee1-dc50-47c3-8046-bda52b7fb672 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.597462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692bee1-dc50-47c3-8046-bda52b7fb672 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.597732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692bee1-dc50-47c3-8046-bda52b7fb672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.609206 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692bee1-dc50-47c3-8046-bda52b7fb672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:46.610710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692bee1-dc50-47c3-8046-bda52b7fb672 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3692bee1-dc50-47c3-8046-bda52b7fb672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:46.627314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:46.628400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:46.629295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:46.629295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:46.629295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:46.629295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 283/570] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:46.629295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692bee1-dc50-47c3-8046-bda52b7fb672 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:46.662046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9482cb5-3383-4a85-b82e-623202c71b7d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:46.704525 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea053f1a-6305-488f-9117-6d9f9dcd32e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.705131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea053f1a-6305-488f-9117-6d9f9dcd32e6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.705406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea053f1a-6305-488f-9117-6d9f9dcd32e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.705669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea053f1a-6305-488f-9117-6d9f9dcd32e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.706589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea053f1a-6305-488f-9117-6d9f9dcd32e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:46.706696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea053f1a-6305-488f-9117-6d9f9dcd32e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:46.707987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea053f1a-6305-488f-9117-6d9f9dcd32e6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:46.717587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 288/571] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:46.723231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cea62-fe8c-4f76-93b4-0f5db424fe1d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.723519 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cea62-fe8c-4f76-93b4-0f5db424fe1d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.723790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cea62-fe8c-4f76-93b4-0f5db424fe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.736371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:46.740281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692bee1-dc50-47c3-8046-bda52b7fb672 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:46.743365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-042cea62-fe8c-4f76-93b4-0f5db424fe1d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:46.744319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-042cea62-fe8c-4f76-93b4-0f5db424fe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:46.769237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:46.777635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9cc81-3782-4a5b-be97-9e43470f5c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.778265 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9cc81-3782-4a5b-be97-9e43470f5c9d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.778541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9cc81-3782-4a5b-be97-9e43470f5c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.779375 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9cc81-3782-4a5b-be97-9e43470f5c9d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.789422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9cc81-3782-4a5b-be97-9e43470f5c9d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:46.790853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9cc81-3782-4a5b-be97-9e43470f5c9d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-edb9cc81-3782-4a5b-be97-9e43470f5c9d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:46.802723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:46.803743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 284/572] 10.222.0.86 () {56 vars in 1093 bytes} [Mon Aug 29 18:48:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:46.804763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9cc81-3782-4a5b-be97-9e43470f5c9d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:46.839098 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 289/573] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:46.839098 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cea62-fe8c-4f76-93b4-0f5db424fe1d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:46.929091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9cc81-3782-4a5b-be97-9e43470f5c9d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:46.944312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cea62-fe8c-4f76-93b4-0f5db424fe1d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:46.969831 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12aaf1-2f09-4e93-a7b4-26c2476e2d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.970591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12aaf1-2f09-4e93-a7b4-26c2476e2d01 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.970958 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12aaf1-2f09-4e93-a7b4-26c2476e2d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.971352 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12aaf1-2f09-4e93-a7b4-26c2476e2d01 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.972162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12aaf1-2f09-4e93-a7b4-26c2476e2d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:46.972565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12aaf1-2f09-4e93-a7b4-26c2476e2d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:46.974058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12aaf1-2f09-4e93-a7b4-26c2476e2d01 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:46.980089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302083bd-4f1b-4454-b710-adef23c78448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:46.980723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302083bd-4f1b-4454-b710-adef23c78448 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:46.980999 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302083bd-4f1b-4454-b710-adef23c78448 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:46.981266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302083bd-4f1b-4454-b710-adef23c78448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:46.987613 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 285/574] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:46.988527 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12aaf1-2f09-4e93-a7b4-26c2476e2d01 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:46.991726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302083bd-4f1b-4454-b710-adef23c78448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:46.993451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302083bd-4f1b-4454-b710-adef23c78448 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-302083bd-4f1b-4454-b710-adef23c78448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:47.009232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:47.011365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:47.015170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:47.015170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:47.015170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:47.015763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:47.015900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 290/575] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:47.015900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302083bd-4f1b-4454-b710-adef23c78448 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:47.043625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:47.100844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12aaf1-2f09-4e93-a7b4-26c2476e2d01 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:47.118838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302083bd-4f1b-4454-b710-adef23c78448 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:47.140329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aded74-398e-4870-94eb-7ae999d0b5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:47.140943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aded74-398e-4870-94eb-7ae999d0b5ae None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:47.141491 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aded74-398e-4870-94eb-7ae999d0b5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:47.141766 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aded74-398e-4870-94eb-7ae999d0b5ae None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:47.142640 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aded74-398e-4870-94eb-7ae999d0b5ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:47.142938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aded74-398e-4870-94eb-7ae999d0b5ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:47.144954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aded74-398e-4870-94eb-7ae999d0b5ae None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:47.154151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276427c-7c12-4465-8fcf-ede4648a475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:47.155049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276427c-7c12-4465-8fcf-ede4648a475b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:47.155477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276427c-7c12-4465-8fcf-ede4648a475b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:47.155924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276427c-7c12-4465-8fcf-ede4648a475b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:47.163667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276427c-7c12-4465-8fcf-ede4648a475b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:47.163861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 286/576] 10.222.0.86 () {56 vars in 1133 bytes} [Mon Aug 29 18:48:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:47.168640 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aded74-398e-4870-94eb-7ae999d0b5ae None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:47.168771 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276427c-7c12-4465-8fcf-ede4648a475b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f276427c-7c12-4465-8fcf-ede4648a475b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:47.180827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:47.181963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:47.183097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:47.183097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:47.183097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:47.183097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:47.183097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:47.183097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:47.183097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:47.183097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:47.183097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:48:47.183097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:47.183097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 291/577] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:47.187699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276427c-7c12-4465-8fcf-ede4648a475b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:47.254029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:47.260278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:47.279744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aded74-398e-4870-94eb-7ae999d0b5ae None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:47.300149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276427c-7c12-4465-8fcf-ede4648a475b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:47.314835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa446994-9a85-418a-8de1-74e43fd3fd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:47.315683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa446994-9a85-418a-8de1-74e43fd3fd27 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:47.315963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa446994-9a85-418a-8de1-74e43fd3fd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:47.316251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa446994-9a85-418a-8de1-74e43fd3fd27 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:47.317000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa446994-9a85-418a-8de1-74e43fd3fd27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:47.317335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa446994-9a85-418a-8de1-74e43fd3fd27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:47.319401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa446994-9a85-418a-8de1-74e43fd3fd27 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:47.331888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 287/578] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:47.332674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa446994-9a85-418a-8de1-74e43fd3fd27 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:47.343947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ad64a-9ed1-4601-81e6-2c0e682877cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:47.344691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ad64a-9ed1-4601-81e6-2c0e682877cf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:47.344893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ad64a-9ed1-4601-81e6-2c0e682877cf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:47.345117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ad64a-9ed1-4601-81e6-2c0e682877cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:47.356195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ad64a-9ed1-4601-81e6-2c0e682877cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4c6ad64a-9ed1-4601-81e6-2c0e682877cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:47.362432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:47.363602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 292/579] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:47.364445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ad64a-9ed1-4601-81e6-2c0e682877cf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:47.441324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa446994-9a85-418a-8de1-74e43fd3fd27 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:47.472467 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ad64a-9ed1-4601-81e6-2c0e682877cf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:47.477384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccee9c2-7569-4d4e-9dbb-e5fbe9f269e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:47.478009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccee9c2-7569-4d4e-9dbb-e5fbe9f269e4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:47.478284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccee9c2-7569-4d4e-9dbb-e5fbe9f269e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:47.478548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccee9c2-7569-4d4e-9dbb-e5fbe9f269e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:47.481317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:47.491528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccee9c2-7569-4d4e-9dbb-e5fbe9f269e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:47.493314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccee9c2-7569-4d4e-9dbb-e5fbe9f269e4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:47.503592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:47.509761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a1576-91ab-4cdb-8ad0-46af735bfcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:47.510366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a1576-91ab-4cdb-8ad0-46af735bfcb2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:47.510641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a1576-91ab-4cdb-8ad0-46af735bfcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:47.510907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a1576-91ab-4cdb-8ad0-46af735bfcb2 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:47.511651 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a1576-91ab-4cdb-8ad0-46af735bfcb2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:47.511952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a1576-91ab-4cdb-8ad0-46af735bfcb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-eccee9c2-7569-4d4e-9dbb-e5fbe9f269e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:47.512046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:47.512975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:47.513935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:47.513935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:47.513935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:47.513935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 288/580] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:47.514645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccee9c2-7569-4d4e-9dbb-e5fbe9f269e4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:47.515238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a1576-91ab-4cdb-8ad0-46af735bfcb2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:47.535899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 293/581] 10.222.0.86 () {56 vars in 1137 bytes} [Mon Aug 29 18:48:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:47.537022 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a1576-91ab-4cdb-8ad0-46af735bfcb2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:47.620919 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccee9c2-7569-4d4e-9dbb-e5fbe9f269e4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:47.647534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a1576-91ab-4cdb-8ad0-46af735bfcb2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:47.654826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e39e-ad13-4649-aad8-a78ee17da02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:47.655491 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e39e-ad13-4649-aad8-a78ee17da02b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:47.655770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e39e-ad13-4649-aad8-a78ee17da02b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:47.656042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e39e-ad13-4649-aad8-a78ee17da02b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:47.663430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849e39e-ad13-4649-aad8-a78ee17da02b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:47.664867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849e39e-ad13-4649-aad8-a78ee17da02b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0849e39e-ad13-4649-aad8-a78ee17da02b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:47.676732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:47.677698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:47.678634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:47.678634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:47.678634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:47.678634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:47.678634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:47.678634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:47.678634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:47.678634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:47.678634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:48:47.678634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:47.678634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 289/582] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:47.679250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e39e-ad13-4649-aad8-a78ee17da02b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:47.693493 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e61afb-ac09-4103-b30e-bd6eada8a998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:47.694382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e61afb-ac09-4103-b30e-bd6eada8a998 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:47.694648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e61afb-ac09-4103-b30e-bd6eada8a998 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:47.694913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e61afb-ac09-4103-b30e-bd6eada8a998 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:47.695964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e61afb-ac09-4103-b30e-bd6eada8a998 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:47.696264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e61afb-ac09-4103-b30e-bd6eada8a998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:47.697939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e61afb-ac09-4103-b30e-bd6eada8a998 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:47.710534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 294/583] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:47.710534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e61afb-ac09-4103-b30e-bd6eada8a998 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:47.741937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:47.760828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:47.797488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e39e-ad13-4649-aad8-a78ee17da02b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:47.824806 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e61afb-ac09-4103-b30e-bd6eada8a998 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:47.847664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962bdd4-f8d0-4201-a70b-2279e8c7b438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:47.848561 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962bdd4-f8d0-4201-a70b-2279e8c7b438 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:47.848561 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962bdd4-f8d0-4201-a70b-2279e8c7b438 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:47.848808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962bdd4-f8d0-4201-a70b-2279e8c7b438 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:47.849422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962bdd4-f8d0-4201-a70b-2279e8c7b438 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:47.849795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962bdd4-f8d0-4201-a70b-2279e8c7b438 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:47.851094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962bdd4-f8d0-4201-a70b-2279e8c7b438 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:47.864566 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c61e4d-80c4-44f3-95a1-5a523397c5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:47.870778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 290/584] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:47.871025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c61e4d-80c4-44f3-95a1-5a523397c5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:47.871025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c61e4d-80c4-44f3-95a1-5a523397c5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:47.871025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c61e4d-80c4-44f3-95a1-5a523397c5a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:47.871417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962bdd4-f8d0-4201-a70b-2279e8c7b438 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:47.886344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c61e4d-80c4-44f3-95a1-5a523397c5a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:47.887833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c61e4d-80c4-44f3-95a1-5a523397c5a0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-d4c61e4d-80c4-44f3-95a1-5a523397c5a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:47.902918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:47.904465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:47.905406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:47.905406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:47.905406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:47.905406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 295/585] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:47.906413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c61e4d-80c4-44f3-95a1-5a523397c5a0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:47.988518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962bdd4-f8d0-4201-a70b-2279e8c7b438 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:47.997890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:48.019129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c61e4d-80c4-44f3-95a1-5a523397c5a0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:48.021368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:48.024210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42289666-4b8a-4637-84fb-3901ac404325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.024960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42289666-4b8a-4637-84fb-3901ac404325 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.025232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42289666-4b8a-4637-84fb-3901ac404325 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.025507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42289666-4b8a-4637-84fb-3901ac404325 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.026343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42289666-4b8a-4637-84fb-3901ac404325 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:48.026651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42289666-4b8a-4637-84fb-3901ac404325 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.028200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42289666-4b8a-4637-84fb-3901ac404325 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.052112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 291/586] 10.222.0.86 () {56 vars in 1137 bytes} [Mon Aug 29 18:48:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:48.052112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42289666-4b8a-4637-84fb-3901ac404325 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:48.065619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06bd41-f23e-441b-bcdc-448ba7dccd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.066280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06bd41-f23e-441b-bcdc-448ba7dccd35 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.066537 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06bd41-f23e-441b-bcdc-448ba7dccd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.066812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06bd41-f23e-441b-bcdc-448ba7dccd35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.079502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06bd41-f23e-441b-bcdc-448ba7dccd35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.081278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06bd41-f23e-441b-bcdc-448ba7dccd35 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fe06bd41-f23e-441b-bcdc-448ba7dccd35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:48.098052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:48.099078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:48.099974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:48.099974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:48.099974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:48.099974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 296/587] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:48.099974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06bd41-f23e-441b-bcdc-448ba7dccd35 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:48.160787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42289666-4b8a-4637-84fb-3901ac404325 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:48.204587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850cae7-caf3-4b49-ac53-7aab22d35815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.205614 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850cae7-caf3-4b49-ac53-7aab22d35815 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.206192 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850cae7-caf3-4b49-ac53-7aab22d35815 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.206758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850cae7-caf3-4b49-ac53-7aab22d35815 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.209502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06bd41-f23e-441b-bcdc-448ba7dccd35 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:48.214050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850cae7-caf3-4b49-ac53-7aab22d35815 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.215797 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850cae7-caf3-4b49-ac53-7aab22d35815 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0850cae7-caf3-4b49-ac53-7aab22d35815 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:48.228891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:48.229807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:48.230712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:48.230712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:48.230712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:48.230712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:48.230712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:48.230712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:48.230712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:48.230712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:48.232247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 292/588] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:48.233042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850cae7-caf3-4b49-ac53-7aab22d35815 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:48.253485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84761444-2cfc-4ff5-9c21-5677ad40e4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.254146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84761444-2cfc-4ff5-9c21-5677ad40e4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.254430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84761444-2cfc-4ff5-9c21-5677ad40e4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.254706 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84761444-2cfc-4ff5-9c21-5677ad40e4c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.255447 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84761444-2cfc-4ff5-9c21-5677ad40e4c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:48.255753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84761444-2cfc-4ff5-9c21-5677ad40e4c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.257224 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84761444-2cfc-4ff5-9c21-5677ad40e4c8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.264156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:48.275804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 297/589] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:48.275804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84761444-2cfc-4ff5-9c21-5677ad40e4c8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:48.293376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:48.345596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850cae7-caf3-4b49-ac53-7aab22d35815 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:48.381045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ba2c7-8a5c-4bf1-b078-158bd41d01ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.381674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ba2c7-8a5c-4bf1-b078-158bd41d01ee None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.381953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ba2c7-8a5c-4bf1-b078-158bd41d01ee None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.382229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ba2c7-8a5c-4bf1-b078-158bd41d01ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.392548 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84761444-2cfc-4ff5-9c21-5677ad40e4c8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:48.393024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ba2c7-8a5c-4bf1-b078-158bd41d01ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.394477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ba2c7-8a5c-4bf1-b078-158bd41d01ee None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e80ba2c7-8a5c-4bf1-b078-158bd41d01ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:48.412377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:48.413332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:48.414201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:48.414201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:48.414201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:48.414201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 293/590] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:48.414201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ba2c7-8a5c-4bf1-b078-158bd41d01ee None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:48.439879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94924e3f-3e29-47a7-a219-f85891b5bfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.440504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94924e3f-3e29-47a7-a219-f85891b5bfd0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.440755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94924e3f-3e29-47a7-a219-f85891b5bfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.441023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94924e3f-3e29-47a7-a219-f85891b5bfd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.441936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94924e3f-3e29-47a7-a219-f85891b5bfd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:48.444102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94924e3f-3e29-47a7-a219-f85891b5bfd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.445466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94924e3f-3e29-47a7-a219-f85891b5bfd0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.457610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 298/591] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:48.458403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94924e3f-3e29-47a7-a219-f85891b5bfd0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:48.501570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:48.524857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ba2c7-8a5c-4bf1-b078-158bd41d01ee None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:48.535506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:48.562211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94924e3f-3e29-47a7-a219-f85891b5bfd0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:48.562574 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b3ee3-20b7-4916-9879-8c3e28a0244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.563221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3ee3-20b7-4916-9879-8c3e28a0244e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.563497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3ee3-20b7-4916-9879-8c3e28a0244e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.563873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3ee3-20b7-4916-9879-8c3e28a0244e None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.564567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b3ee3-20b7-4916-9879-8c3e28a0244e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:48.564860 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b3ee3-20b7-4916-9879-8c3e28a0244e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.566342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b3ee3-20b7-4916-9879-8c3e28a0244e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.579712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 294/592] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:48.580537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b3ee3-20b7-4916-9879-8c3e28a0244e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:48.598952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b5a45-b45c-4978-b6ad-dca882e8c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.599610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b5a45-b45c-4978-b6ad-dca882e8c326 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.599890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b5a45-b45c-4978-b6ad-dca882e8c326 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.600170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b5a45-b45c-4978-b6ad-dca882e8c326 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.606800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b5a45-b45c-4978-b6ad-dca882e8c326 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.608270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b5a45-b45c-4978-b6ad-dca882e8c326 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-801b5a45-b45c-4978-b6ad-dca882e8c326 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:48.619428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:48.620344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:48.621292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:48.621292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:48.621292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:48.621292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:48.621292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:48.621292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:48.621292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:48.621292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:48.621292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 299/593] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:48.623318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902a5cf-f104-421a-9c57-c0b82d5fb016 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.623318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902a5cf-f104-421a-9c57-c0b82d5fb016 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.623318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902a5cf-f104-421a-9c57-c0b82d5fb016 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.627047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 300/594] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:48.627047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902a5cf-f104-421a-9c57-c0b82d5fb016 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:48.691828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b3ee3-20b7-4916-9879-8c3e28a0244e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:48.727369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897938e-7897-4c82-9f95-90546e39ffcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.727985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897938e-7897-4c82-9f95-90546e39ffcc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.728265 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897938e-7897-4c82-9f95-90546e39ffcc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.728538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897938e-7897-4c82-9f95-90546e39ffcc None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.729183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897938e-7897-4c82-9f95-90546e39ffcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:48.729488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897938e-7897-4c82-9f95-90546e39ffcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.730861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897938e-7897-4c82-9f95-90546e39ffcc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.738076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:48.739536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902a5cf-f104-421a-9c57-c0b82d5fb016 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:48.744941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 295/595] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:48.745717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897938e-7897-4c82-9f95-90546e39ffcc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:48.773592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af4c40-eeb0-40a6-b564-3ab57ed4048f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.774298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4c40-eeb0-40a6-b564-3ab57ed4048f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.774663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4c40-eeb0-40a6-b564-3ab57ed4048f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.774943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4c40-eeb0-40a6-b564-3ab57ed4048f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.776880 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:48.785152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af4c40-eeb0-40a6-b564-3ab57ed4048f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.786587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af4c40-eeb0-40a6-b564-3ab57ed4048f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-03af4c40-eeb0-40a6-b564-3ab57ed4048f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:48.798970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:48.800075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 301/596] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:48.801561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af4c40-eeb0-40a6-b564-3ab57ed4048f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:48.853096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897938e-7897-4c82-9f95-90546e39ffcc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:48.885808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d7c69-29f6-46eb-87de-d767df03ff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.886544 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d7c69-29f6-46eb-87de-d767df03ff56 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.886828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d7c69-29f6-46eb-87de-d767df03ff56 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.887132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d7c69-29f6-46eb-87de-d767df03ff56 None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/045b22d046c549f2b53d2ed73b94f936/roles/000d8f9e15c142349b2f85e8ca441e96` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.901746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d7c69-29f6-46eb-87de-d767df03ff56 None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=045b22d046c549f2b53d2ed73b94f936, role_id=000d8f9e15c142349b2f85e8ca441e96)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.903227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d7c69-29f6-46eb-87de-d767df03ff56 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.905305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af4c40-eeb0-40a6-b564-3ab57ed4048f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:48.916168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 296/597] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:48:48 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/045b22d046c549f2b53d2ed73b94f936/roles/000d8f9e15c142349b2f85e8ca441e96 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:48:48.917367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5a815-1892-4857-ac4e-9920c86b443f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.917644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5a815-1892-4857-ac4e-9920c86b443f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.917917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5a815-1892-4857-ac4e-9920c86b443f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.937229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-72a5a815-1892-4857-ac4e-9920c86b443f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:48.938552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-72a5a815-1892-4857-ac4e-9920c86b443f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:48.941658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4c804-5db4-476a-b948-ffe50cd9f827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:48.942278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c804-5db4-476a-b948-ffe50cd9f827 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:48.942623 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c804-5db4-476a-b948-ffe50cd9f827 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:48.942825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c804-5db4-476a-b948-ffe50cd9f827 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:48.943561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4c804-5db4-476a-b948-ffe50cd9f827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:48.944115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4c804-5db4-476a-b948-ffe50cd9f827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:48.945556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4c804-5db4-476a-b948-ffe50cd9f827 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:48.960296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 302/598] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:48.961152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4c804-5db4-476a-b948-ffe50cd9f827 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:48.992983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:49.023492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:49.024137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 297/599] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:49.025134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5a815-1892-4857-ac4e-9920c86b443f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:49.089233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4c804-5db4-476a-b948-ffe50cd9f827 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:49.131463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97988652-a841-47e6-bdaa-11954aefb0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:49.132455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97988652-a841-47e6-bdaa-11954aefb0cf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:49.132733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97988652-a841-47e6-bdaa-11954aefb0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:49.133311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97988652-a841-47e6-bdaa-11954aefb0cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:49.141217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97988652-a841-47e6-bdaa-11954aefb0cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:49.142674 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97988652-a841-47e6-bdaa-11954aefb0cf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-97988652-a841-47e6-bdaa-11954aefb0cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:49.155996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:49.156923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:49.157811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:49.157811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:49.157811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:49.157811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:49.157811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:49.157811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:49.157811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:49.157811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:49.157811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 303/600] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:49.158302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5a815-1892-4857-ac4e-9920c86b443f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:49.158375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97988652-a841-47e6-bdaa-11954aefb0cf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:49.197501 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f1976-9a0d-444a-ac53-95cb5bfb4a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:49.198008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f1976-9a0d-444a-ac53-95cb5bfb4a33 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:49.198293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f1976-9a0d-444a-ac53-95cb5bfb4a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:49.200901 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f1976-9a0d-444a-ac53-95cb5bfb4a33 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:49.200901 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f1976-9a0d-444a-ac53-95cb5bfb4a33 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:49.200901 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f1976-9a0d-444a-ac53-95cb5bfb4a33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:49.201515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f1976-9a0d-444a-ac53-95cb5bfb4a33 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:49.222861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 298/601] 10.222.0.86 () {56 vars in 1139 bytes} [Mon Aug 29 18:48:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:49.228096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f1976-9a0d-444a-ac53-95cb5bfb4a33 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:49.266290 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:49.292788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97988652-a841-47e6-bdaa-11954aefb0cf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:49.312159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:49.335096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658ef98-455a-4965-b3e0-1773156ffdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:49.335815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658ef98-455a-4965-b3e0-1773156ffdee None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:49.336032 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658ef98-455a-4965-b3e0-1773156ffdee None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:49.336303 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658ef98-455a-4965-b3e0-1773156ffdee None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/a3030e2c49614aee892c7693da6b73bf/roles/000d8f9e15c142349b2f85e8ca441e96` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:49.343780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f1976-9a0d-444a-ac53-95cb5bfb4a33 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:49.351545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658ef98-455a-4965-b3e0-1773156ffdee None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=a3030e2c49614aee892c7693da6b73bf, role_id=000d8f9e15c142349b2f85e8ca441e96)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:49.352982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658ef98-455a-4965-b3e0-1773156ffdee None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:49.367111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 304/602] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:48:49 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/a3030e2c49614aee892c7693da6b73bf/roles/000d8f9e15c142349b2f85e8ca441e96 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:48:49.367923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658ef98-455a-4965-b3e0-1773156ffdee None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:49.385348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf8d20-29be-43cf-a401-66941e2ef92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:49.386117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf8d20-29be-43cf-a401-66941e2ef92c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:49.386466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf8d20-29be-43cf-a401-66941e2ef92c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:49.386845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf8d20-29be-43cf-a401-66941e2ef92c None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:49.387741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf8d20-29be-43cf-a401-66941e2ef92c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:49.388853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf8d20-29be-43cf-a401-66941e2ef92c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:49.390453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf8d20-29be-43cf-a401-66941e2ef92c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:49.408286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 299/603] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:49.408286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf8d20-29be-43cf-a401-66941e2ef92c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:49.512187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:49.520254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf8d20-29be-43cf-a401-66941e2ef92c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:49.527929 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658ef98-455a-4965-b3e0-1773156ffdee None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:49.567988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeead16-3d6b-4618-af21-8936c0a46470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:49.568871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeead16-3d6b-4618-af21-8936c0a46470 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:49.569152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeead16-3d6b-4618-af21-8936c0a46470 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:49.569421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeead16-3d6b-4618-af21-8936c0a46470 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:49.577003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8e9ed-e415-4398-b238-81afc496b121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:49.577657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8e9ed-e415-4398-b238-81afc496b121 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:49.577948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8e9ed-e415-4398-b238-81afc496b121 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:49.578223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8e9ed-e415-4398-b238-81afc496b121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:49.583183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeead16-3d6b-4618-af21-8936c0a46470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-eeeead16-3d6b-4618-af21-8936c0a46470 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:49.589734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:49.590688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:49.591595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:49.592952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8e9ed-e415-4398-b238-81afc496b121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:49.594654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8e9ed-e415-4398-b238-81afc496b121 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:49.595169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 300/604] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:49.595984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeead16-3d6b-4618-af21-8936c0a46470 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-afd8e9ed-e415-4398-b238-81afc496b121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:49.611709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:49.612703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:49.613605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:49.613605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:49.613605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:49.615673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 305/605] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:49.616657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8e9ed-e415-4398-b238-81afc496b121 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:49.645470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:49.713021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeead16-3d6b-4618-af21-8936c0a46470 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:49.750438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34bb43-491c-4d0e-9965-eff86a83807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:49.751476 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34bb43-491c-4d0e-9965-eff86a83807e None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:49.751968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34bb43-491c-4d0e-9965-eff86a83807e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:49.752498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34bb43-491c-4d0e-9965-eff86a83807e None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/9af69888b04c45b09e34d9ff7a0b564f/roles/000d8f9e15c142349b2f85e8ca441e96` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:49.761976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8e9ed-e415-4398-b238-81afc496b121 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:49.768814 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34bb43-491c-4d0e-9965-eff86a83807e None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=9af69888b04c45b09e34d9ff7a0b564f, role_id=000d8f9e15c142349b2f85e8ca441e96)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:49.770528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34bb43-491c-4d0e-9965-eff86a83807e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:49.798340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 301/606] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:48:49 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/9af69888b04c45b09e34d9ff7a0b564f/roles/000d8f9e15c142349b2f85e8ca441e96 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:48:49.799485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34bb43-491c-4d0e-9965-eff86a83807e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:49.802918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24db02-eed7-4dc2-a8f1-723f4c8dbdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:49.803758 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24db02-eed7-4dc2-a8f1-723f4c8dbdb5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:49.804040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24db02-eed7-4dc2-a8f1-723f4c8dbdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:49.804428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24db02-eed7-4dc2-a8f1-723f4c8dbdb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:49.805110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24db02-eed7-4dc2-a8f1-723f4c8dbdb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:49.805421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24db02-eed7-4dc2-a8f1-723f4c8dbdb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:49.806826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24db02-eed7-4dc2-a8f1-723f4c8dbdb5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:49.812018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:49.825039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 306/607] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:49.827314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24db02-eed7-4dc2-a8f1-723f4c8dbdb5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:49.936654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:49.962869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24db02-eed7-4dc2-a8f1-723f4c8dbdb5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:49.989041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34bb43-491c-4d0e-9965-eff86a83807e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:50.009163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d648f-ecc7-4ed6-81b8-52438cb4a307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:50.010112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d648f-ecc7-4ed6-81b8-52438cb4a307 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:50.010905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d648f-ecc7-4ed6-81b8-52438cb4a307 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:50.011388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d648f-ecc7-4ed6-81b8-52438cb4a307 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:50.025495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d648f-ecc7-4ed6-81b8-52438cb4a307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:50.027164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d648f-ecc7-4ed6-81b8-52438cb4a307 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-677d648f-ecc7-4ed6-81b8-52438cb4a307 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:50.043529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:50.045148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:50.046308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:50.046308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:50.046308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:50.046490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fab51-33ab-41aa-b5fe-cd27f7b9f84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:50.046490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fab51-33ab-41aa-b5fe-cd27f7b9f84b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:50.046490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fab51-33ab-41aa-b5fe-cd27f7b9f84b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:50.046793 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fab51-33ab-41aa-b5fe-cd27f7b9f84b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:50.047913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fab51-33ab-41aa-b5fe-cd27f7b9f84b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:50.048284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fab51-33ab-41aa-b5fe-cd27f7b9f84b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:50.050319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fab51-33ab-41aa-b5fe-cd27f7b9f84b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:50.050521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 307/608] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:50.055397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d648f-ecc7-4ed6-81b8-52438cb4a307 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:50.063768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 302/609] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:50.069690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fab51-33ab-41aa-b5fe-cd27f7b9f84b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:50.155420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:50.173116 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d648f-ecc7-4ed6-81b8-52438cb4a307 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:50.182883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fab51-33ab-41aa-b5fe-cd27f7b9f84b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:50.203436 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:50.221198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462b89c-4597-42ab-aaa3-5804c58a47c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:50.221851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462b89c-4597-42ab-aaa3-5804c58a47c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:50.222118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462b89c-4597-42ab-aaa3-5804c58a47c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:50.222393 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462b89c-4597-42ab-aaa3-5804c58a47c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:50.227703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25cb86-6795-4d23-bfc0-a2b5e583fc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:50.228582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25cb86-6795-4d23-bfc0-a2b5e583fc5c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:50.229065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25cb86-6795-4d23-bfc0-a2b5e583fc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:50.229521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462b89c-4597-42ab-aaa3-5804c58a47c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:50.229750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25cb86-6795-4d23-bfc0-a2b5e583fc5c None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:50.230603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25cb86-6795-4d23-bfc0-a2b5e583fc5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:50.231046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462b89c-4597-42ab-aaa3-5804c58a47c4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:50.231323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25cb86-6795-4d23-bfc0-a2b5e583fc5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:50.233382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25cb86-6795-4d23-bfc0-a2b5e583fc5c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6462b89c-4597-42ab-aaa3-5804c58a47c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:50.243854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:50.244963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:50.246049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:50.246049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:50.246049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:50.246049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:50.246049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:50.246049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:50.246049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:50.246049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:50.248656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 303/610] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:50.259036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462b89c-4597-42ab-aaa3-5804c58a47c4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:50.263500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 308/611] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:50.269064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25cb86-6795-4d23-bfc0-a2b5e583fc5c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:50.371114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462b89c-4597-42ab-aaa3-5804c58a47c4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:50.401531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e102e6d-a9a8-42c9-9877-459337cbf96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:50.402331 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e102e6d-a9a8-42c9-9877-459337cbf96a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:50.402711 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e102e6d-a9a8-42c9-9877-459337cbf96a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:50.403136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e102e6d-a9a8-42c9-9877-459337cbf96a None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:50.404743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e102e6d-a9a8-42c9-9877-459337cbf96a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:50.406072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e102e6d-a9a8-42c9-9877-459337cbf96a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:50.407799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e102e6d-a9a8-42c9-9877-459337cbf96a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:50.411365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:50.431844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 304/612] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:50.451144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e102e6d-a9a8-42c9-9877-459337cbf96a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:50.469780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25cb86-6795-4d23-bfc0-a2b5e583fc5c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:50.541608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ddc4b-668a-4951-bd0c-afb55cd98d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:50.544228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ddc4b-668a-4951-bd0c-afb55cd98d5d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:50.544985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ddc4b-668a-4951-bd0c-afb55cd98d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:50.546386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ddc4b-668a-4951-bd0c-afb55cd98d5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:50.564857 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ddc4b-668a-4951-bd0c-afb55cd98d5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:50.566532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ddc4b-668a-4951-bd0c-afb55cd98d5d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f97ddc4b-668a-4951-bd0c-afb55cd98d5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:50.589487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:50.590613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:50.592454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:50.592454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:50.592454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:50.592454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:50.592454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:50.592454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:50.592454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:50.592454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:50.592454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:48:50.592454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:50.600547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 309/613] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:48:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:50.606334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ddc4b-668a-4951-bd0c-afb55cd98d5d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:50.626270 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e102e6d-a9a8-42c9-9877-459337cbf96a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:50.663454 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f6ffd-c00c-46c0-bff8-cac69540b73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:50.664257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f6ffd-c00c-46c0-bff8-cac69540b73d None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:50.664636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f6ffd-c00c-46c0-bff8-cac69540b73d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:50.665112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f6ffd-c00c-46c0-bff8-cac69540b73d None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/5ec64944b34d4ef1bd666bb1b7e216da/roles/000d8f9e15c142349b2f85e8ca441e96` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:50.680567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f6ffd-c00c-46c0-bff8-cac69540b73d None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=5ec64944b34d4ef1bd666bb1b7e216da, role_id=000d8f9e15c142349b2f85e8ca441e96)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:50.682214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f6ffd-c00c-46c0-bff8-cac69540b73d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:50.712938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:50.714954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 305/614] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:48:50 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/5ec64944b34d4ef1bd666bb1b7e216da/roles/000d8f9e15c142349b2f85e8ca441e96 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:48:50.775479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ddc4b-668a-4951-bd0c-afb55cd98d5d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:50.824729 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38780df-989a-4a60-b716-6d3d86ad1f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:50.826218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38780df-989a-4a60-b716-6d3d86ad1f88 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:50.826680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38780df-989a-4a60-b716-6d3d86ad1f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:50.827119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38780df-989a-4a60-b716-6d3d86ad1f88 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:50.828059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38780df-989a-4a60-b716-6d3d86ad1f88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:50.828534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38780df-989a-4a60-b716-6d3d86ad1f88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:50.830137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38780df-989a-4a60-b716-6d3d86ad1f88 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:50.862489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 310/615] 10.222.0.86 () {56 vars in 1135 bytes} [Mon Aug 29 18:48:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:50.868655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f6ffd-c00c-46c0-bff8-cac69540b73d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:50.913163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:50.986566 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f6ffd-c00c-46c0-bff8-cac69540b73d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:51.023802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fc731-d26a-465d-8fe4-ceeb6b5a35f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:51.024449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fc731-d26a-465d-8fe4-ceeb6b5a35f6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:51.024729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fc731-d26a-465d-8fe4-ceeb6b5a35f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:51.025004 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fc731-d26a-465d-8fe4-ceeb6b5a35f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:51.035891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fc731-d26a-465d-8fe4-ceeb6b5a35f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:51.037382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fc731-d26a-465d-8fe4-ceeb6b5a35f6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-411fc731-d26a-465d-8fe4-ceeb6b5a35f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:51.052933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:51.053955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:51.055139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:51.055139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:51.055139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:51.055139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 306/616] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:51.060889 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38780df-989a-4a60-b716-6d3d86ad1f88 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:51.184586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38780df-989a-4a60-b716-6d3d86ad1f88 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:51.195264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:51.222474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dcb10-46ce-4bce-8e29-ca56040dea3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:51.223116 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dcb10-46ce-4bce-8e29-ca56040dea3f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:51.223400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dcb10-46ce-4bce-8e29-ca56040dea3f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:51.223675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dcb10-46ce-4bce-8e29-ca56040dea3f None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:51.224326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dcb10-46ce-4bce-8e29-ca56040dea3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:51.224631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dcb10-46ce-4bce-8e29-ca56040dea3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:51.225986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dcb10-46ce-4bce-8e29-ca56040dea3f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:51.238658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 311/617] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:51.244346 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fc731-d26a-465d-8fe4-ceeb6b5a35f6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:51.345115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:51.357791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fc731-d26a-465d-8fe4-ceeb6b5a35f6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:51.393158 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb246b4-80bd-4736-bc17-49962bc73fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:51.393802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb246b4-80bd-4736-bc17-49962bc73fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:51.394079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb246b4-80bd-4736-bc17-49962bc73fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:51.394411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb246b4-80bd-4736-bc17-49962bc73fc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:51.401627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb246b4-80bd-4736-bc17-49962bc73fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:51.403105 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb246b4-80bd-4736-bc17-49962bc73fc5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8bb246b4-80bd-4736-bc17-49962bc73fc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:51.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:51.416445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:51.417477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:51.417477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:51.417477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:51.417477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:51.417477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:51.417477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:51.417477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:51.417477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:51.417477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 307/618] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:51.428131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dcb10-46ce-4bce-8e29-ca56040dea3f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:51.558863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dcb10-46ce-4bce-8e29-ca56040dea3f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:51.589001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93256e4c-0ec5-4f40-8edb-06e41977d724 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:51.589314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93256e4c-0ec5-4f40-8edb-06e41977d724 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:51.589545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93256e4c-0ec5-4f40-8edb-06e41977d724 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:51.591098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 308/619] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:48:51.596340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d7b07-baa3-4385-97f7-73da51ca30fb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:51.596897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d7b07-baa3-4385-97f7-73da51ca30fb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:51.597402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d7b07-baa3-4385-97f7-73da51ca30fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:51.608931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe29a7b-8909-48e4-94eb-1df02fa406ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:51.609549 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe29a7b-8909-48e4-94eb-1df02fa406ac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:51.609832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe29a7b-8909-48e4-94eb-1df02fa406ac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:51.610108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe29a7b-8909-48e4-94eb-1df02fa406ac None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:51.610875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe29a7b-8909-48e4-94eb-1df02fa406ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:51.611216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe29a7b-8909-48e4-94eb-1df02fa406ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:51.613352 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe29a7b-8909-48e4-94eb-1df02fa406ac None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:51.620961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c59d7b07-baa3-4385-97f7-73da51ca30fb None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:51.624276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c59d7b07-baa3-4385-97f7-73da51ca30fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:51.641865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 312/620] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:51.647322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe29a7b-8909-48e4-94eb-1df02fa406ac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:51.696383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:51.717277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 309/621] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:51.718760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913abd2-6f78-4765-a004-2890ad3db712 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:51.719089 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913abd2-6f78-4765-a004-2890ad3db712 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:51.719365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913abd2-6f78-4765-a004-2890ad3db712 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:51.720848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 310/622] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:51.732514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccc31-0a1d-4191-86b0-93ad189d9fab None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:51.732803 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccc31-0a1d-4191-86b0-93ad189d9fab None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:51.733076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccc31-0a1d-4191-86b0-93ad189d9fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:51.754870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-eefccc31-0a1d-4191-86b0-93ad189d9fab None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:51.756061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eefccc31-0a1d-4191-86b0-93ad189d9fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:51.833980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe29a7b-8909-48e4-94eb-1df02fa406ac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:51.856408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 311/623] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:48:51.877383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8474f09-e83b-45d3-9354-d0cf9db4f1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:51.878103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8474f09-e83b-45d3-9354-d0cf9db4f1f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:51.878433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8474f09-e83b-45d3-9354-d0cf9db4f1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:51.878763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8474f09-e83b-45d3-9354-d0cf9db4f1f6 None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/dd1b7173843b4da48e69da8b36d59226/roles/000d8f9e15c142349b2f85e8ca441e96` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:51.893527 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8474f09-e83b-45d3-9354-d0cf9db4f1f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=dd1b7173843b4da48e69da8b36d59226, role_id=000d8f9e15c142349b2f85e8ca441e96)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:51.895150 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8474f09-e83b-45d3-9354-d0cf9db4f1f6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:51.927945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 313/624] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:48:51 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/dd1b7173843b4da48e69da8b36d59226/roles/000d8f9e15c142349b2f85e8ca441e96 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:48:52.079198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefccc31-0a1d-4191-86b0-93ad189d9fab None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:52.091252 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:52.197427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefccc31-0a1d-4191-86b0-93ad189d9fab None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:52.222675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8474f09-e83b-45d3-9354-d0cf9db4f1f6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:52.243284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe176e-f111-4349-ae70-d232561fac00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:52.243947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe176e-f111-4349-ae70-d232561fac00 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:52.244228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe176e-f111-4349-ae70-d232561fac00 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:52.244521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe176e-f111-4349-ae70-d232561fac00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:52.259259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe176e-f111-4349-ae70-d232561fac00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c5fe176e-f111-4349-ae70-d232561fac00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:52.265719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:52.266671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:52.267571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 312/625] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:52.268790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d5065-d011-4885-8efe-2d635fd154d1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:52.268790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d5065-d011-4885-8efe-2d635fd154d1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:52.269597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d5065-d011-4885-8efe-2d635fd154d1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:52.274401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 313/626] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:52.275233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d5065-d011-4885-8efe-2d635fd154d1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:52.318968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:52.349316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:52.351292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8474f09-e83b-45d3-9354-d0cf9db4f1f6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:52.392923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d5065-d011-4885-8efe-2d635fd154d1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:52.393146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dee90-fc61-4904-a6ac-4abcafe12849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:52.393733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dee90-fc61-4904-a6ac-4abcafe12849 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:52.394014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dee90-fc61-4904-a6ac-4abcafe12849 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:52.394389 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dee90-fc61-4904-a6ac-4abcafe12849 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:52.406130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6dee90-fc61-4904-a6ac-4abcafe12849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-7f6dee90-fc61-4904-a6ac-4abcafe12849 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:52.411406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:52.412306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:52.413141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 314/627] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:52.414311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b076d9-a71a-43ae-a831-71f275bb4c6a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:52.414468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b076d9-a71a-43ae-a831-71f275bb4c6a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:52.417536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b076d9-a71a-43ae-a831-71f275bb4c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:52.435899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-71b076d9-a71a-43ae-a831-71f275bb4c6a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:52.437226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-71b076d9-a71a-43ae-a831-71f275bb4c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:52.439825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7a787-64cc-4fc5-9715-188a63d3c430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:52.441327 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7a787-64cc-4fc5-9715-188a63d3c430 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:52.441612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7a787-64cc-4fc5-9715-188a63d3c430 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:52.441890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7a787-64cc-4fc5-9715-188a63d3c430 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:52.442561 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7a787-64cc-4fc5-9715-188a63d3c430 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:52.442866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7a787-64cc-4fc5-9715-188a63d3c430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:52.444664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7a787-64cc-4fc5-9715-188a63d3c430 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:52.455634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 314/628] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:52.456535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7a787-64cc-4fc5-9715-188a63d3c430 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:52.515999 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 315/629] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:52.516930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b076d9-a71a-43ae-a831-71f275bb4c6a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:52.609767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:52.636468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7a787-64cc-4fc5-9715-188a63d3c430 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:52.667396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:52.674912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebdfc8-4a9c-4c9a-9702-2a2798403eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:52.680805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebdfc8-4a9c-4c9a-9702-2a2798403eef None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:52.681095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebdfc8-4a9c-4c9a-9702-2a2798403eef None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:52.681378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebdfc8-4a9c-4c9a-9702-2a2798403eef None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:52.682059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebdfc8-4a9c-4c9a-9702-2a2798403eef None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:52.682422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebdfc8-4a9c-4c9a-9702-2a2798403eef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:52.688937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebdfc8-4a9c-4c9a-9702-2a2798403eef None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:52.689065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b076d9-a71a-43ae-a831-71f275bb4c6a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:52.698481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 315/630] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:52.707982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebdfc8-4a9c-4c9a-9702-2a2798403eef None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:52.733768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ebd37-e08c-4b43-bdcc-cc0d2b14ec26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:52.734461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ebd37-e08c-4b43-bdcc-cc0d2b14ec26 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:52.734742 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ebd37-e08c-4b43-bdcc-cc0d2b14ec26 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:52.735089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ebd37-e08c-4b43-bdcc-cc0d2b14ec26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:52.746427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ebd37-e08c-4b43-bdcc-cc0d2b14ec26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:52.747994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ebd37-e08c-4b43-bdcc-cc0d2b14ec26 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-247ebd37-e08c-4b43-bdcc-cc0d2b14ec26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:52.763901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:52.764859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:52.765896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:52.765896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:52.765896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:52.765896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 316/631] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:52.774750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ebd37-e08c-4b43-bdcc-cc0d2b14ec26 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:52.882084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebdfc8-4a9c-4c9a-9702-2a2798403eef None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:52.890877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ebd37-e08c-4b43-bdcc-cc0d2b14ec26 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:52.912850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:52.927286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5f97a-7956-48fb-ab80-9ec2c00376ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:52.928588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5f97a-7956-48fb-ab80-9ec2c00376ee None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:52.928866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5f97a-7956-48fb-ab80-9ec2c00376ee None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:52.929135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5f97a-7956-48fb-ab80-9ec2c00376ee None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:52.929779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5f97a-7956-48fb-ab80-9ec2c00376ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:52.930083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5f97a-7956-48fb-ab80-9ec2c00376ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:52.931478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5f97a-7956-48fb-ab80-9ec2c00376ee None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:52.937399 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a102b-b1fa-4bb5-b1b5-55473f424ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:52.938774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a102b-b1fa-4bb5-b1b5-55473f424ffb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:52.939485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a102b-b1fa-4bb5-b1b5-55473f424ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:52.940710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a102b-b1fa-4bb5-b1b5-55473f424ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:52.945670 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 317/632] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:52.946499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5f97a-7956-48fb-ab80-9ec2c00376ee None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:52.956142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a102b-b1fa-4bb5-b1b5-55473f424ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:52.958825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a102b-b1fa-4bb5-b1b5-55473f424ffb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8f3a102b-b1fa-4bb5-b1b5-55473f424ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:52.979601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:52.980542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:52.981428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:52.981428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:52.981428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:52.984948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 316/633] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:52.985883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a102b-b1fa-4bb5-b1b5-55473f424ffb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:53.046857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:53.061438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5f97a-7956-48fb-ab80-9ec2c00376ee None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:53.100472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009f47d-ba5a-481f-b495-e2d15d9289bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:53.101104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009f47d-ba5a-481f-b495-e2d15d9289bd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:53.101387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009f47d-ba5a-481f-b495-e2d15d9289bd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:53.101660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009f47d-ba5a-481f-b495-e2d15d9289bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:53.103173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a102b-b1fa-4bb5-b1b5-55473f424ffb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:53.112262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009f47d-ba5a-481f-b495-e2d15d9289bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2009f47d-ba5a-481f-b495-e2d15d9289bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:53.117375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:53.118286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 318/634] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:53.119528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009f47d-ba5a-481f-b495-e2d15d9289bd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:53.145845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bccb6d5-b98b-4cbc-a837-babb751c0d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:53.146397 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccb6d5-b98b-4cbc-a837-babb751c0d71 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:53.146677 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccb6d5-b98b-4cbc-a837-babb751c0d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:53.146950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccb6d5-b98b-4cbc-a837-babb751c0d71 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:53.155380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccb6d5-b98b-4cbc-a837-babb751c0d71 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:53.156903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccb6d5-b98b-4cbc-a837-babb751c0d71 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3bccb6d5-b98b-4cbc-a837-babb751c0d71 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:53.168202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:53.169248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:53.170248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:53.170248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:53.170248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:53.170248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:53.170248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:53.170248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:53.170248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:53.170248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:53.170248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:48:53.170248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:53.170811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:53.170903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 317/635] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:53.170903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba5c54-9fd4-4e33-a7b2-530d5abd9c5c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:53.171071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba5c54-9fd4-4e33-a7b2-530d5abd9c5c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:53.171300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba5c54-9fd4-4e33-a7b2-530d5abd9c5c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:53.172756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 318/636] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:53.173354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba5c54-9fd4-4e33-a7b2-530d5abd9c5c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:53.243559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009f47d-ba5a-481f-b495-e2d15d9289bd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:53.281787 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1495f0-00ad-4f84-889d-8dceed043551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:53.282485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1495f0-00ad-4f84-889d-8dceed043551 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:53.282898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1495f0-00ad-4f84-889d-8dceed043551 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:53.283173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba5c54-9fd4-4e33-a7b2-530d5abd9c5c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:53.283605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1495f0-00ad-4f84-889d-8dceed043551 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:53.283886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1495f0-00ad-4f84-889d-8dceed043551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:53.284190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1495f0-00ad-4f84-889d-8dceed043551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:53.285555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1495f0-00ad-4f84-889d-8dceed043551 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:53.286334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:53.302515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 319/637] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:53.302515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1495f0-00ad-4f84-889d-8dceed043551 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:53.329326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae266165-55fc-4e76-9ac5-00cf02e0c44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:53.330026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae266165-55fc-4e76-9ac5-00cf02e0c44d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:53.330459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae266165-55fc-4e76-9ac5-00cf02e0c44d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:53.330743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae266165-55fc-4e76-9ac5-00cf02e0c44d None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:53.331538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae266165-55fc-4e76-9ac5-00cf02e0c44d None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:53.331959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae266165-55fc-4e76-9ac5-00cf02e0c44d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:53.333427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae266165-55fc-4e76-9ac5-00cf02e0c44d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:53.346946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 319/638] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:53.348370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeee56-aaff-4263-8599-b89a7caf6ce6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:53.348662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeee56-aaff-4263-8599-b89a7caf6ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:53.349057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeee56-aaff-4263-8599-b89a7caf6ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:53.368562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ebfeee56-aaff-4263-8599-b89a7caf6ce6 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:53.369754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ebfeee56-aaff-4263-8599-b89a7caf6ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:53.406795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:53.410662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1495f0-00ad-4f84-889d-8dceed043551 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:53.438484 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 320/639] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:53.439356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfeee56-aaff-4263-8599-b89a7caf6ce6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:53.455337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f70c1-adc3-47eb-99e5-737eb89cc9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:53.456496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f70c1-adc3-47eb-99e5-737eb89cc9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:53.457437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f70c1-adc3-47eb-99e5-737eb89cc9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:53.458053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f70c1-adc3-47eb-99e5-737eb89cc9d7 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:53.459115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f70c1-adc3-47eb-99e5-737eb89cc9d7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:53.459826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f70c1-adc3-47eb-99e5-737eb89cc9d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:53.461791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f70c1-adc3-47eb-99e5-737eb89cc9d7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:53.484327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 320/640] 10.222.0.86 () {56 vars in 1137 bytes} [Mon Aug 29 18:48:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:53.485273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f70c1-adc3-47eb-99e5-737eb89cc9d7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:53.561898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:53.570474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfeee56-aaff-4263-8599-b89a7caf6ce6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:53.601431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f70c1-adc3-47eb-99e5-737eb89cc9d7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:53.615776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60251f2a-266c-414a-93f6-13cd23caab54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:53.616828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60251f2a-266c-414a-93f6-13cd23caab54 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:53.617269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60251f2a-266c-414a-93f6-13cd23caab54 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:53.618069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60251f2a-266c-414a-93f6-13cd23caab54 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:53.626941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60251f2a-266c-414a-93f6-13cd23caab54 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:53.629372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60251f2a-266c-414a-93f6-13cd23caab54 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:53.640365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee685862-84cf-46a5-8d0e-9b8126e14166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:53.641177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee685862-84cf-46a5-8d0e-9b8126e14166 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:53.641554 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee685862-84cf-46a5-8d0e-9b8126e14166 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:53.641926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee685862-84cf-46a5-8d0e-9b8126e14166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-60251f2a-266c-414a-93f6-13cd23caab54 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:53.645280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:53.646408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:53.647603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:53.647603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:53.647603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:53.647603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:53.647603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:53.647603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:53.647603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:53.647603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:53.647603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:48:53.647603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:53.649449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 321/641] 10.222.0.86 () {56 vars in 1093 bytes} [Mon Aug 29 18:48:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:53.650515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60251f2a-266c-414a-93f6-13cd23caab54 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:53.657602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee685862-84cf-46a5-8d0e-9b8126e14166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:53.659207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee685862-84cf-46a5-8d0e-9b8126e14166 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ee685862-84cf-46a5-8d0e-9b8126e14166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:53.677404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:53.678662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:53.679813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:53.679813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:53.679813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:53.685809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 321/642] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:53.686897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee685862-84cf-46a5-8d0e-9b8126e14166 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:53.702005 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:53.777513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60251f2a-266c-414a-93f6-13cd23caab54 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:53.805000 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee685862-84cf-46a5-8d0e-9b8126e14166 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:53.817764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f2aee-7cb1-4d51-b2c1-1ea3f695b8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:53.818377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f2aee-7cb1-4d51-b2c1-1ea3f695b8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:53.818913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f2aee-7cb1-4d51-b2c1-1ea3f695b8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:53.819226 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f2aee-7cb1-4d51-b2c1-1ea3f695b8f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:53.840469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710068ff-2a64-4c15-a4f6-fada317f8da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:53.841610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710068ff-2a64-4c15-a4f6-fada317f8da5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:53.841892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710068ff-2a64-4c15-a4f6-fada317f8da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:53.842484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710068ff-2a64-4c15-a4f6-fada317f8da5 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:53.842889 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710068ff-2a64-4c15-a4f6-fada317f8da5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:53.843223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710068ff-2a64-4c15-a4f6-fada317f8da5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:53.844648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710068ff-2a64-4c15-a4f6-fada317f8da5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:53.846519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:53.869663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 322/643] 10.222.0.86 () {56 vars in 1133 bytes} [Mon Aug 29 18:48:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:53.869663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710068ff-2a64-4c15-a4f6-fada317f8da5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:53.901351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f2aee-7cb1-4d51-b2c1-1ea3f695b8f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:53.904578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f2aee-7cb1-4d51-b2c1-1ea3f695b8f5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:53.943524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:53.982713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710068ff-2a64-4c15-a4f6-fada317f8da5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:54.019406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa9e66-886b-4a22-9bb9-3077ab41f297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:54.020021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9e66-886b-4a22-9bb9-3077ab41f297 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.020303 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9e66-886b-4a22-9bb9-3077ab41f297 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.020578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9e66-886b-4a22-9bb9-3077ab41f297 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.021228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa9e66-886b-4a22-9bb9-3077ab41f297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:54.021535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa9e66-886b-4a22-9bb9-3077ab41f297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:54.027039 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 322/644] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:54.027182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa9e66-886b-4a22-9bb9-3077ab41f297 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:54.027287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f2aee-7cb1-4d51-b2c1-1ea3f695b8f5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:54.040098 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 323/645] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:54.045908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa9e66-886b-4a22-9bb9-3077ab41f297 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:54.087999 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:54.133368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f2aee-7cb1-4d51-b2c1-1ea3f695b8f5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:54.157626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa9e66-886b-4a22-9bb9-3077ab41f297 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:54.169103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52817689-ec48-4f80-9e3e-8c9b7a30799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:54.169734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52817689-ec48-4f80-9e3e-8c9b7a30799e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.170010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52817689-ec48-4f80-9e3e-8c9b7a30799e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.170278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52817689-ec48-4f80-9e3e-8c9b7a30799e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.180401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52817689-ec48-4f80-9e3e-8c9b7a30799e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:54.183939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-52817689-ec48-4f80-9e3e-8c9b7a30799e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:54.185272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:54.186227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 323/646] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:54.187848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52817689-ec48-4f80-9e3e-8c9b7a30799e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:54.196866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda923a-7d82-448a-97d8-0e16f106f13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:54.197422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda923a-7d82-448a-97d8-0e16f106f13c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.197702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda923a-7d82-448a-97d8-0e16f106f13c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.197974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda923a-7d82-448a-97d8-0e16f106f13c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.273335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda923a-7d82-448a-97d8-0e16f106f13c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:54.275174 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda923a-7d82-448a-97d8-0e16f106f13c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:54.293356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52817689-ec48-4f80-9e3e-8c9b7a30799e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:54.313731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:54.333474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce387f2-fa0d-4fc3-8926-f81df7c7f3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:54.334095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce387f2-fa0d-4fc3-8926-f81df7c7f3ef None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.334378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce387f2-fa0d-4fc3-8926-f81df7c7f3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.334644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce387f2-fa0d-4fc3-8926-f81df7c7f3ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.345246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce387f2-fa0d-4fc3-8926-f81df7c7f3ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:54.346601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce387f2-fa0d-4fc3-8926-f81df7c7f3ef None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8ce387f2-fa0d-4fc3-8926-f81df7c7f3ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:54.360935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:54.361971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:54.362873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:54.362873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:54.362873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:54.362873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 324/647] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:54.362873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce387f2-fa0d-4fc3-8926-f81df7c7f3ef None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:54.384865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 324/648] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:54.384865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bacac2-6af4-4361-ba45-63074c0efacc None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.384865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bacac2-6af4-4361-ba45-63074c0efacc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.384865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bacac2-6af4-4361-ba45-63074c0efacc None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.384865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 325/649] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:54.384865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bacac2-6af4-4361-ba45-63074c0efacc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:54.411857 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:54.468302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce387f2-fa0d-4fc3-8926-f81df7c7f3ef None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:54.485480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bacac2-6af4-4361-ba45-63074c0efacc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:54.502289 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07917588-d3fd-4d96-8e8f-98b8e780e3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:54.502915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07917588-d3fd-4d96-8e8f-98b8e780e3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.503232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07917588-d3fd-4d96-8e8f-98b8e780e3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.503500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07917588-d3fd-4d96-8e8f-98b8e780e3b7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.510648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07917588-d3fd-4d96-8e8f-98b8e780e3b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:54.512111 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07917588-d3fd-4d96-8e8f-98b8e780e3b7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:54.519327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e57dae-fa4e-4715-a97e-a857d5dd6fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:54.519926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e57dae-fa4e-4715-a97e-a857d5dd6fca None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.520200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e57dae-fa4e-4715-a97e-a857d5dd6fca None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.520469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e57dae-fa4e-4715-a97e-a857d5dd6fca None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.521121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e57dae-fa4e-4715-a97e-a857d5dd6fca None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:54.521419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e57dae-fa4e-4715-a97e-a857d5dd6fca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:54.522770 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e57dae-fa4e-4715-a97e-a857d5dd6fca None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-07917588-d3fd-4d96-8e8f-98b8e780e3b7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:54.523498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:54.524388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 325/650] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:54.525575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07917588-d3fd-4d96-8e8f-98b8e780e3b7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:54.533167 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 326/651] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:54.533968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e57dae-fa4e-4715-a97e-a857d5dd6fca None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:54.556079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:54.632656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:54.634208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07917588-d3fd-4d96-8e8f-98b8e780e3b7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:54.645861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e57dae-fa4e-4715-a97e-a857d5dd6fca None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:54.671642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4483f-11d4-4aba-8806-38086d08879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:54.672192 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4483f-11d4-4aba-8806-38086d08879c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.672467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4483f-11d4-4aba-8806-38086d08879c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.672734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4483f-11d4-4aba-8806-38086d08879c None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.673447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4483f-11d4-4aba-8806-38086d08879c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:54.673921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4483f-11d4-4aba-8806-38086d08879c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:54.676120 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4483f-11d4-4aba-8806-38086d08879c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:54.683107 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eeeba-06ae-4e87-a508-119304cddab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:54.683743 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeeba-06ae-4e87-a508-119304cddab0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.684026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeeba-06ae-4e87-a508-119304cddab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.684300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeeba-06ae-4e87-a508-119304cddab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.693412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 326/652] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:54.694918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd81269-0bb3-4e51-898a-ceea786988ce None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.695237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd81269-0bb3-4e51-898a-ceea786988ce None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.695516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd81269-0bb3-4e51-898a-ceea786988ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.703400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eeeba-06ae-4e87-a508-119304cddab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:54.704848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eeeba-06ae-4e87-a508-119304cddab0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:54.714453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fbd81269-0bb3-4e51-898a-ceea786988ce None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:54.715695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fbd81269-0bb3-4e51-898a-ceea786988ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-818eeeba-06ae-4e87-a508-119304cddab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:54.719821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:54.720753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:54.721656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:54.721656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:54.721656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:54.721656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 327/653] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:54.721656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eeeba-06ae-4e87-a508-119304cddab0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:54.785344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 327/654] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:54.786190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd81269-0bb3-4e51-898a-ceea786988ce None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:54.803628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:54.830899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eeeba-06ae-4e87-a508-119304cddab0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:54.876530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c892848-4e64-4b4d-bd2a-d921217ab62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:54.877937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c892848-4e64-4b4d-bd2a-d921217ab62c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.878437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c892848-4e64-4b4d-bd2a-d921217ab62c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.879166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c892848-4e64-4b4d-bd2a-d921217ab62c None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.880353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c892848-4e64-4b4d-bd2a-d921217ab62c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:54.881272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c892848-4e64-4b4d-bd2a-d921217ab62c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:54.883407 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c892848-4e64-4b4d-bd2a-d921217ab62c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:54.891217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd81269-0bb3-4e51-898a-ceea786988ce None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:54.899370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:54.903471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 328/655] 10.222.0.86 () {56 vars in 1137 bytes} [Mon Aug 29 18:48:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:54.904456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c892848-4e64-4b4d-bd2a-d921217ab62c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:54.935876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d7c4d-a12c-4b30-b1cc-cf3e4412265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:54.936670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d7c4d-a12c-4b30-b1cc-cf3e4412265c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:54.936953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d7c4d-a12c-4b30-b1cc-cf3e4412265c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:54.937554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d7c4d-a12c-4b30-b1cc-cf3e4412265c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:54.950115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d7c4d-a12c-4b30-b1cc-cf3e4412265c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:54.952333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d7c4d-a12c-4b30-b1cc-cf3e4412265c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b18d7c4d-a12c-4b30-b1cc-cf3e4412265c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:54.970128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:54.971072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:54.971956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:54.971956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:54.971956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:54.971956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 328/656] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:54.971956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d7c4d-a12c-4b30-b1cc-cf3e4412265c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.015556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c892848-4e64-4b4d-bd2a-d921217ab62c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:55.053394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a2f62-6ff3-479d-aae5-851ea93fb133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:55.054017 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2f62-6ff3-479d-aae5-851ea93fb133 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:55.054294 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2f62-6ff3-479d-aae5-851ea93fb133 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:55.054562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2f62-6ff3-479d-aae5-851ea93fb133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:55.061850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a2f62-6ff3-479d-aae5-851ea93fb133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:55.063584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a2f62-6ff3-479d-aae5-851ea93fb133 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:55.069113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-d94a2f62-6ff3-479d-aae5-851ea93fb133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:55.075148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:55.076169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:55.077102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:55.077102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:55.077102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:55.077102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:55.077102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:55.077102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:55.077102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:55.077102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:55.077102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 329/657] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:55.077600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a2f62-6ff3-479d-aae5-851ea93fb133 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.103817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d7c4d-a12c-4b30-b1cc-cf3e4412265c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:55.137248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850148e-3cfe-4c22-9590-36d8629e7372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:55.137872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850148e-3cfe-4c22-9590-36d8629e7372 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:55.138145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850148e-3cfe-4c22-9590-36d8629e7372 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:55.138412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850148e-3cfe-4c22-9590-36d8629e7372 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:55.139074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850148e-3cfe-4c22-9590-36d8629e7372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:55.139200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:55.139374 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850148e-3cfe-4c22-9590-36d8629e7372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:55.140883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850148e-3cfe-4c22-9590-36d8629e7372 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:55.152629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 329/658] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:55.153610 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850148e-3cfe-4c22-9590-36d8629e7372 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.182468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a2f62-6ff3-479d-aae5-851ea93fb133 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:55.216820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4f5c2-da41-4f0f-ad4f-a23aa40451ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:55.217421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4f5c2-da41-4f0f-ad4f-a23aa40451ea None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:55.217705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4f5c2-da41-4f0f-ad4f-a23aa40451ea None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:55.218002 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4f5c2-da41-4f0f-ad4f-a23aa40451ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:55.252809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850148e-3cfe-4c22-9590-36d8629e7372 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:55.281477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:55.284417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4f5c2-da41-4f0f-ad4f-a23aa40451ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:55.285856 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4f5c2-da41-4f0f-ad4f-a23aa40451ea None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:55.287545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172352f-a373-46ed-829b-abe69f5590ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:55.288141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172352f-a373-46ed-829b-abe69f5590ed None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:55.288425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172352f-a373-46ed-829b-abe69f5590ed None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:55.288698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172352f-a373-46ed-829b-abe69f5590ed None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:55.289361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172352f-a373-46ed-829b-abe69f5590ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:55.289667 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172352f-a373-46ed-829b-abe69f5590ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:55.291045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172352f-a373-46ed-829b-abe69f5590ed None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:55.303441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 330/659] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:55.304742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172352f-a373-46ed-829b-abe69f5590ed None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.345793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:55.389342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 330/660] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:55.390341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4f5c2-da41-4f0f-ad4f-a23aa40451ea None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.413066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172352f-a373-46ed-829b-abe69f5590ed None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:55.448759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bc799-4eec-4a06-8c2c-efda80af9573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:55.449392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bc799-4eec-4a06-8c2c-efda80af9573 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:55.449669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bc799-4eec-4a06-8c2c-efda80af9573 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:55.449944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bc799-4eec-4a06-8c2c-efda80af9573 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:55.460508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bc799-4eec-4a06-8c2c-efda80af9573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9f8bc799-4eec-4a06-8c2c-efda80af9573 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:55.465692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:55.467321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 331/661] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:55.468314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bc799-4eec-4a06-8c2c-efda80af9573 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.509265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4f5c2-da41-4f0f-ad4f-a23aa40451ea None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:55.519647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:55.542647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75017e7c-350c-41a0-a730-854565e70ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:55.543271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75017e7c-350c-41a0-a730-854565e70ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:55.543549 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75017e7c-350c-41a0-a730-854565e70ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:55.543818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75017e7c-350c-41a0-a730-854565e70ed4 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:55.545205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75017e7c-350c-41a0-a730-854565e70ed4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:55.545503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75017e7c-350c-41a0-a730-854565e70ed4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:55.546911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75017e7c-350c-41a0-a730-854565e70ed4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:55.558294 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 331/662] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:55.559113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75017e7c-350c-41a0-a730-854565e70ed4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.576358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bc799-4eec-4a06-8c2c-efda80af9573 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:55.580760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:55.609290 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4063b3-7efa-4379-a415-f1aec8daa36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:55.609912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4063b3-7efa-4379-a415-f1aec8daa36b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:55.610188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4063b3-7efa-4379-a415-f1aec8daa36b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:55.610460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4063b3-7efa-4379-a415-f1aec8daa36b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:55.617664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4063b3-7efa-4379-a415-f1aec8daa36b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:55.619128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4063b3-7efa-4379-a415-f1aec8daa36b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4e4063b3-7efa-4379-a415-f1aec8daa36b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:55.630311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:55.631384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:55.632423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:55.632423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:55.632423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:55.632423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:55.632423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:55.632423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:55.632423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:55.632423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:55.632423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 332/663] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:55.632916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4063b3-7efa-4379-a415-f1aec8daa36b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.662155 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75017e7c-350c-41a0-a730-854565e70ed4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:55.697747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d89bae-410c-49e1-a8fe-71a079321baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:55.698375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d89bae-410c-49e1-a8fe-71a079321baa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:55.698654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d89bae-410c-49e1-a8fe-71a079321baa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:55.698933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d89bae-410c-49e1-a8fe-71a079321baa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:55.707635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d89bae-410c-49e1-a8fe-71a079321baa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:55.709062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d89bae-410c-49e1-a8fe-71a079321baa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-51d89bae-410c-49e1-a8fe-71a079321baa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:55.720592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:55.721535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 332/664] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:48:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:55.722476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d89bae-410c-49e1-a8fe-71a079321baa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.738915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4063b3-7efa-4379-a415-f1aec8daa36b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:55.750255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:55.774477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b50bc-14b0-4956-86a8-af4533eaf633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:55.775049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b50bc-14b0-4956-86a8-af4533eaf633 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:55.775329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b50bc-14b0-4956-86a8-af4533eaf633 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:55.775605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b50bc-14b0-4956-86a8-af4533eaf633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:55.786705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b50bc-14b0-4956-86a8-af4533eaf633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:55.788585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b50bc-14b0-4956-86a8-af4533eaf633 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e81b50bc-14b0-4956-86a8-af4533eaf633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:55.803903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:55.804961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:55.805851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:55.805851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:55.805851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:55.805851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 333/665] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:55.805851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b50bc-14b0-4956-86a8-af4533eaf633 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.810078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:55.830331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d89bae-410c-49e1-a8fe-71a079321baa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:55.864100 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35e1dd-a58b-45a3-834e-4cbcbddaafc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:55.864700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e1dd-a58b-45a3-834e-4cbcbddaafc2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:55.864978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e1dd-a58b-45a3-834e-4cbcbddaafc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:55.865255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e1dd-a58b-45a3-834e-4cbcbddaafc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:55.865910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35e1dd-a58b-45a3-834e-4cbcbddaafc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:55.866214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35e1dd-a58b-45a3-834e-4cbcbddaafc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:55.867665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35e1dd-a58b-45a3-834e-4cbcbddaafc2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:55.875614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 333/666] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:48:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:55.876596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35e1dd-a58b-45a3-834e-4cbcbddaafc2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.906498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b50bc-14b0-4956-86a8-af4533eaf633 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:55.938967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f6704-e254-4752-af11-d92c104f0f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:55.939591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f6704-e254-4752-af11-d92c104f0f67 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:55.939887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f6704-e254-4752-af11-d92c104f0f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:55.940151 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f6704-e254-4752-af11-d92c104f0f67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:55.940865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f6704-e254-4752-af11-d92c104f0f67 None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=a3030e2c49614aee892c7693da6b73bf) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:55.941174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f6704-e254-4752-af11-d92c104f0f67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:55.942594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f6704-e254-4752-af11-d92c104f0f67 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:55.948565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 334/667] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:55 2022] GET /identity/v3/role_assignments?user.id=a3030e2c49614aee892c7693da6b73bf&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:55.949341 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f6704-e254-4752-af11-d92c104f0f67 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:55.980046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:55.993306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35e1dd-a58b-45a3-834e-4cbcbddaafc2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:56.032326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83f3cb-1681-406a-a142-e26f763475d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:56.033325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83f3cb-1681-406a-a142-e26f763475d2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:56.033805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83f3cb-1681-406a-a142-e26f763475d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:56.034275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83f3cb-1681-406a-a142-e26f763475d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:56.035383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83f3cb-1681-406a-a142-e26f763475d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:56.035864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83f3cb-1681-406a-a142-e26f763475d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:56.037559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83f3cb-1681-406a-a142-e26f763475d2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:56.051141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 334/668] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:56.051141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83f3cb-1681-406a-a142-e26f763475d2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:56.051141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:56.071033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f6704-e254-4752-af11-d92c104f0f67 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:56.119760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d0ff4-d634-4bc6-a043-a9f40b9b3167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:56.120370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d0ff4-d634-4bc6-a043-a9f40b9b3167 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:56.120653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d0ff4-d634-4bc6-a043-a9f40b9b3167 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:56.120932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d0ff4-d634-4bc6-a043-a9f40b9b3167 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:56.121660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d0ff4-d634-4bc6-a043-a9f40b9b3167 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:56.121964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d0ff4-d634-4bc6-a043-a9f40b9b3167 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:56.123952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d0ff4-d634-4bc6-a043-a9f40b9b3167 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:56.145499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 335/669] 10.222.0.86 () {56 vars in 1139 bytes} [Mon Aug 29 18:48:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:56.146231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d0ff4-d634-4bc6-a043-a9f40b9b3167 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:56.172910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83f3cb-1681-406a-a142-e26f763475d2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:56.210829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877d873-d055-4def-b686-9cee39fc578f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:56.211607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877d873-d055-4def-b686-9cee39fc578f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:56.211988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877d873-d055-4def-b686-9cee39fc578f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:56.212297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877d873-d055-4def-b686-9cee39fc578f None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:56.212948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877d873-d055-4def-b686-9cee39fc578f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:56.213246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877d873-d055-4def-b686-9cee39fc578f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:56.214716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877d873-d055-4def-b686-9cee39fc578f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:56.231051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 335/670] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:56.231051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877d873-d055-4def-b686-9cee39fc578f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:56.240974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:56.258009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d0ff4-d634-4bc6-a043-a9f40b9b3167 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:56.292567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:56.293452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d948c26-f365-4836-800b-44684d68a8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:56.294217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d948c26-f365-4836-800b-44684d68a8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:56.294519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d948c26-f365-4836-800b-44684d68a8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:56.294846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d948c26-f365-4836-800b-44684d68a8c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:56.305210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d948c26-f365-4836-800b-44684d68a8c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:56.306658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d948c26-f365-4836-800b-44684d68a8c1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7d948c26-f365-4836-800b-44684d68a8c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:56.322866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:56.323888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:56.324771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:56.324771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:56.324771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:56.324771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 336/671] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:56.324771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d948c26-f365-4836-800b-44684d68a8c1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:56.343439 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877d873-d055-4def-b686-9cee39fc578f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:56.379640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6153c51-69ac-4764-8b8d-91d45ed59197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:56.380371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6153c51-69ac-4764-8b8d-91d45ed59197 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:56.380576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6153c51-69ac-4764-8b8d-91d45ed59197 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:56.380783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6153c51-69ac-4764-8b8d-91d45ed59197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:56.381509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6153c51-69ac-4764-8b8d-91d45ed59197 None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=045b22d046c549f2b53d2ed73b94f936) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:56.381815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6153c51-69ac-4764-8b8d-91d45ed59197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:56.383381 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6153c51-69ac-4764-8b8d-91d45ed59197 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:56.390244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 336/672] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:56 2022] GET /identity/v3/role_assignments?user.id=045b22d046c549f2b53d2ed73b94f936&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:56.391214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6153c51-69ac-4764-8b8d-91d45ed59197 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:56.466592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d948c26-f365-4836-800b-44684d68a8c1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:56.504737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1934b-ab9a-455e-bc46-000ab1e2bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:56.505357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1934b-ab9a-455e-bc46-000ab1e2bd3d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:56.505634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1934b-ab9a-455e-bc46-000ab1e2bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:56.505903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1934b-ab9a-455e-bc46-000ab1e2bd3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:56.511226 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:56.530308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6153c51-69ac-4764-8b8d-91d45ed59197 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:56.565871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:56.579280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770ded2-7094-4141-9666-5c760821bed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:56.579926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770ded2-7094-4141-9666-5c760821bed9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:56.580206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770ded2-7094-4141-9666-5c760821bed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:56.580478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770ded2-7094-4141-9666-5c760821bed9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:56.591448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770ded2-7094-4141-9666-5c760821bed9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:56.592310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770ded2-7094-4141-9666-5c760821bed9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:56.600439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf1934b-ab9a-455e-bc46-000ab1e2bd3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:56.602082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf1934b-ab9a-455e-bc46-000ab1e2bd3d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c770ded2-7094-4141-9666-5c760821bed9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:56.605654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:56.606616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:56.607953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:56.607953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:56.607953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:56.607953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:56.607953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:56.607953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:56.607953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:56.607953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:56.607953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 337/673] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:56.608436 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770ded2-7094-4141-9666-5c760821bed9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:56.731973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770ded2-7094-4141-9666-5c760821bed9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:56.751664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 337/674] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:56.753022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1934b-ab9a-455e-bc46-000ab1e2bd3d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:56.768645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b063348-1963-4e6e-87f3-e900fbaa066c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:56.769335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b063348-1963-4e6e-87f3-e900fbaa066c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:56.769679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b063348-1963-4e6e-87f3-e900fbaa066c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:56.770014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b063348-1963-4e6e-87f3-e900fbaa066c None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:56.770915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b063348-1963-4e6e-87f3-e900fbaa066c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:56.771081 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b063348-1963-4e6e-87f3-e900fbaa066c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:56.772524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b063348-1963-4e6e-87f3-e900fbaa066c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:56.785667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 338/675] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:56.786603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b063348-1963-4e6e-87f3-e900fbaa066c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:56.799376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:56.860810 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:56.873142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1934b-ab9a-455e-bc46-000ab1e2bd3d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:56.917355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920f04a-36fc-4b9a-b63c-7bfb574520cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:56.917974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920f04a-36fc-4b9a-b63c-7bfb574520cc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:56.918250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920f04a-36fc-4b9a-b63c-7bfb574520cc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:56.918516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920f04a-36fc-4b9a-b63c-7bfb574520cc None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:56.919303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920f04a-36fc-4b9a-b63c-7bfb574520cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:56.919608 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920f04a-36fc-4b9a-b63c-7bfb574520cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:56.921156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920f04a-36fc-4b9a-b63c-7bfb574520cc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:56.933543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b063348-1963-4e6e-87f3-e900fbaa066c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:56.934443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 338/676] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:56.935816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920f04a-36fc-4b9a-b63c-7bfb574520cc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:56.982720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d01ccc-976e-44e9-9438-cb46814e9e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:56.983785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d01ccc-976e-44e9-9438-cb46814e9e2f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:56.984344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d01ccc-976e-44e9-9438-cb46814e9e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:56.984820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d01ccc-976e-44e9-9438-cb46814e9e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:56.996533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d01ccc-976e-44e9-9438-cb46814e9e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:56.998833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d01ccc-976e-44e9-9438-cb46814e9e2f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-48d01ccc-976e-44e9-9438-cb46814e9e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:57.016312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:57.017312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:57.018192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:57.018192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:57.018192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:57.020589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 339/677] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:57.021672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d01ccc-976e-44e9-9438-cb46814e9e2f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:57.043042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:57.049011 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920f04a-36fc-4b9a-b63c-7bfb574520cc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:57.085169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898bdeb-3f68-4ebf-a231-8478b71adf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:57.085796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898bdeb-3f68-4ebf-a231-8478b71adf69 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:57.086071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898bdeb-3f68-4ebf-a231-8478b71adf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:57.086340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898bdeb-3f68-4ebf-a231-8478b71adf69 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:57.094209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898bdeb-3f68-4ebf-a231-8478b71adf69 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:57.095685 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898bdeb-3f68-4ebf-a231-8478b71adf69 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2898bdeb-3f68-4ebf-a231-8478b71adf69 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:48:57.109313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:57.110383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:57.111432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:57.111432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:57.111432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:48:57.111432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:48:57.111432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:48:57.111432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:48:57.111432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:48:57.111432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:48:57.111432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:48:57.111432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:57.115234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 339/678] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:48:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:57.116100 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898bdeb-3f68-4ebf-a231-8478b71adf69 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:57.140854 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:57.145580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d01ccc-976e-44e9-9438-cb46814e9e2f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:57.188443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdab825-af7e-4258-8622-194a9e7bfbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:57.189523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdab825-af7e-4258-8622-194a9e7bfbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:57.190133 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdab825-af7e-4258-8622-194a9e7bfbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:57.190613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdab825-af7e-4258-8622-194a9e7bfbe5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:57.191719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdab825-af7e-4258-8622-194a9e7bfbe5 None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=9af69888b04c45b09e34d9ff7a0b564f) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:57.192249 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdab825-af7e-4258-8622-194a9e7bfbe5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:57.194288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdab825-af7e-4258-8622-194a9e7bfbe5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:57.200912 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 340/679] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:57 2022] GET /identity/v3/role_assignments?user.id=9af69888b04c45b09e34d9ff7a0b564f&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:57.201989 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdab825-af7e-4258-8622-194a9e7bfbe5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:57.246109 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898bdeb-3f68-4ebf-a231-8478b71adf69 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:57.302435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4197f-0575-45ea-b201-d176447630bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:57.303090 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4197f-0575-45ea-b201-d176447630bd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:57.303372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4197f-0575-45ea-b201-d176447630bd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:57.303644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4197f-0575-45ea-b201-d176447630bd None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:57.304391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4197f-0575-45ea-b201-d176447630bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:57.304590 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4197f-0575-45ea-b201-d176447630bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:57.305965 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4197f-0575-45ea-b201-d176447630bd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:57.323920 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdab825-af7e-4258-8622-194a9e7bfbe5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:57.342757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 340/680] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:57.348341 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4197f-0575-45ea-b201-d176447630bd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:57.366006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a90f6-224a-4dc0-ac7f-3c294780d004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:57.366982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a90f6-224a-4dc0-ac7f-3c294780d004 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:57.367516 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a90f6-224a-4dc0-ac7f-3c294780d004 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:57.368092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a90f6-224a-4dc0-ac7f-3c294780d004 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:57.369192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a90f6-224a-4dc0-ac7f-3c294780d004 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:57.369724 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a90f6-224a-4dc0-ac7f-3c294780d004 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:57.371580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a90f6-224a-4dc0-ac7f-3c294780d004 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:57.385336 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:57.395188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 341/681] 10.222.0.86 () {56 vars in 1135 bytes} [Mon Aug 29 18:48:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:57.401035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a90f6-224a-4dc0-ac7f-3c294780d004 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:57.427837 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:57.524396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a90f6-224a-4dc0-ac7f-3c294780d004 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:57.528082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4197f-0575-45ea-b201-d176447630bd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:57.565051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820249c-c258-4a2f-8777-3c25ce31a097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:57.565692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820249c-c258-4a2f-8777-3c25ce31a097 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:57.565975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820249c-c258-4a2f-8777-3c25ce31a097 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:57.566266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820249c-c258-4a2f-8777-3c25ce31a097 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:57.570020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a39f1-af3e-4f75-bb34-33bdad15e761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:57.570906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a39f1-af3e-4f75-bb34-33bdad15e761 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:57.571432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a39f1-af3e-4f75-bb34-33bdad15e761 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:57.571907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a39f1-af3e-4f75-bb34-33bdad15e761 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:57.579483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a39f1-af3e-4f75-bb34-33bdad15e761 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:57.581307 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a39f1-af3e-4f75-bb34-33bdad15e761 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-176a39f1-af3e-4f75-bb34-33bdad15e761 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:57.593925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:57.594915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:57.595889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:57.595889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:57.595889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:57.595889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:57.595889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:57.595889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:57.595889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:57.595889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:57.598398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 341/682] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:57.604170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a39f1-af3e-4f75-bb34-33bdad15e761 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:57.649883 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6820249c-c258-4a2f-8777-3c25ce31a097 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:57.651549 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6820249c-c258-4a2f-8777-3c25ce31a097 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:57.688452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:57.713886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:57.744830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a39f1-af3e-4f75-bb34-33bdad15e761 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:57.789924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034efb9-d934-436a-88b4-98d34108e505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:57.790902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034efb9-d934-436a-88b4-98d34108e505 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:57.791436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034efb9-d934-436a-88b4-98d34108e505 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:57.791916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034efb9-d934-436a-88b4-98d34108e505 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:57.792820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034efb9-d934-436a-88b4-98d34108e505 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:57.793353 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034efb9-d934-436a-88b4-98d34108e505 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:57.795024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034efb9-d934-436a-88b4-98d34108e505 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:57.795318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 342/683] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:48:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:48:57.797011 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820249c-c258-4a2f-8777-3c25ce31a097 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:57.810241 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 342/684] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:57.816046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034efb9-d934-436a-88b4-98d34108e505 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:57.920824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820249c-c258-4a2f-8777-3c25ce31a097 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:57.945410 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:57.965595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7679fa-5f4e-435c-9967-25fb7c3df53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:57.971756 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7679fa-5f4e-435c-9967-25fb7c3df53b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:57.972040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7679fa-5f4e-435c-9967-25fb7c3df53b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:57.972319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7679fa-5f4e-435c-9967-25fb7c3df53b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:57.990530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034efb9-d934-436a-88b4-98d34108e505 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:58.000133 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7679fa-5f4e-435c-9967-25fb7c3df53b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:58.001553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7679fa-5f4e-435c-9967-25fb7c3df53b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4d7679fa-5f4e-435c-9967-25fb7c3df53b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:58.032658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:58.033699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:58.034584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:58.034584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:58.034584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:58.038605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 343/685] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:58.039467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8ac72-19fe-409f-8d3a-3f442dcdb588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:58.040321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8ac72-19fe-409f-8d3a-3f442dcdb588 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:58.040801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8ac72-19fe-409f-8d3a-3f442dcdb588 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:58.041270 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8ac72-19fe-409f-8d3a-3f442dcdb588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:58.042342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8ac72-19fe-409f-8d3a-3f442dcdb588 None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=5ec64944b34d4ef1bd666bb1b7e216da) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:58.042849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8ac72-19fe-409f-8d3a-3f442dcdb588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:58.045484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7679fa-5f4e-435c-9967-25fb7c3df53b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:58.046187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8ac72-19fe-409f-8d3a-3f442dcdb588 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:58.053242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:58.063627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 343/686] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:57 2022] GET /identity/v3/role_assignments?user.id=5ec64944b34d4ef1bd666bb1b7e216da&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:48:58.136478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cde1ad-1f5d-452a-9973-ec7359d93933 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:58.136825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cde1ad-1f5d-452a-9973-ec7359d93933 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:58.137025 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cde1ad-1f5d-452a-9973-ec7359d93933 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:58.138421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 344/687] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:48:58.155866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a9684-b68b-4fcb-a31e-7c9ee3946a36 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:58.156209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a9684-b68b-4fcb-a31e-7c9ee3946a36 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:58.156415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a9684-b68b-4fcb-a31e-7c9ee3946a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:58.171076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7679fa-5f4e-435c-9967-25fb7c3df53b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:58.197122 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-577a9684-b68b-4fcb-a31e-7c9ee3946a36 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:58.198364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-577a9684-b68b-4fcb-a31e-7c9ee3946a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:58.232490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b473c73-786c-4a8f-9cd1-4ae11a2b6ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:58.233497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b473c73-786c-4a8f-9cd1-4ae11a2b6ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:58.233983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b473c73-786c-4a8f-9cd1-4ae11a2b6ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:58.234610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b473c73-786c-4a8f-9cd1-4ae11a2b6ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:58.243029 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b473c73-786c-4a8f-9cd1-4ae11a2b6ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:58.243877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b473c73-786c-4a8f-9cd1-4ae11a2b6ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:58.245520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b473c73-786c-4a8f-9cd1-4ae11a2b6ae6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:58.273233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 344/688] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:58.279510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b473c73-786c-4a8f-9cd1-4ae11a2b6ae6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:58.301684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 345/689] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:58.323893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:58.394772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b473c73-786c-4a8f-9cd1-4ae11a2b6ae6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:58.431629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464bbb2-f396-41eb-b1b3-b0b05be9b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:58.432181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464bbb2-f396-41eb-b1b3-b0b05be9b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:58.432459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464bbb2-f396-41eb-b1b3-b0b05be9b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:58.432728 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464bbb2-f396-41eb-b1b3-b0b05be9b4d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:58.439776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464bbb2-f396-41eb-b1b3-b0b05be9b4d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:58.441228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464bbb2-f396-41eb-b1b3-b0b05be9b4d5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a464bbb2-f396-41eb-b1b3-b0b05be9b4d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:48:58.453160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:58.454094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:58.454969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:48:58.454969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:48:58.454969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:48:58.454969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:48:58.454969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:48:58.454969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:48:58.454969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:48:58.454969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:58.454969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 345/690] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:48:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:58.460211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a9684-b68b-4fcb-a31e-7c9ee3946a36 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:58.562107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a9684-b68b-4fcb-a31e-7c9ee3946a36 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:58.595654 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:58.603728 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464bbb2-f396-41eb-b1b3-b0b05be9b4d5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:58.604738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2838c66-d254-4f8b-9311-c6dd0cd1b905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:58.605385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2838c66-d254-4f8b-9311-c6dd0cd1b905 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:58.605657 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2838c66-d254-4f8b-9311-c6dd0cd1b905 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:58.605928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2838c66-d254-4f8b-9311-c6dd0cd1b905 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:58.606910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2838c66-d254-4f8b-9311-c6dd0cd1b905 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:58.607246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2838c66-d254-4f8b-9311-c6dd0cd1b905 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:58.608702 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2838c66-d254-4f8b-9311-c6dd0cd1b905 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:58.621240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 346/691] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:48:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:58.624683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3f7b2-6dff-495d-a5a3-0d213db6d8f6 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:58.625218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3f7b2-6dff-495d-a5a3-0d213db6d8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:58.625642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3f7b2-6dff-495d-a5a3-0d213db6d8f6 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:58.627364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 347/692] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:48:58.628072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3f7b2-6dff-495d-a5a3-0d213db6d8f6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:58.725364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:58.736179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464bbb2-f396-41eb-b1b3-b0b05be9b4d5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:58.770557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3f7b2-6dff-495d-a5a3-0d213db6d8f6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:58.774170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649f3d8-8a2b-4d99-851d-e0e5182d5730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:58.774804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649f3d8-8a2b-4d99-851d-e0e5182d5730 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:58.775120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649f3d8-8a2b-4d99-851d-e0e5182d5730 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:58.775394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649f3d8-8a2b-4d99-851d-e0e5182d5730 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:58.775977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649f3d8-8a2b-4d99-851d-e0e5182d5730 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:58.777333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649f3d8-8a2b-4d99-851d-e0e5182d5730 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-7649f3d8-8a2b-4d99-851d-e0e5182d5730 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:58.789392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:58.790637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:48:58.791580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 346/693] 10.222.0.86 () {56 vars in 1129 bytes} [Mon Aug 29 18:48:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:48:58.792366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5971a2-35e6-4de3-99d2-04f5d6bcf023 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:58.792438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5971a2-35e6-4de3-99d2-04f5d6bcf023 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:58.792685 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5971a2-35e6-4de3-99d2-04f5d6bcf023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:58.814149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fa5971a2-35e6-4de3-99d2-04f5d6bcf023 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:58.815415 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fa5971a2-35e6-4de3-99d2-04f5d6bcf023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:48:58.816509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9db17-2fc3-412d-8b35-843f659daba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:58.817285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9db17-2fc3-412d-8b35-843f659daba2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:58.818027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9db17-2fc3-412d-8b35-843f659daba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:58.819523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9db17-2fc3-412d-8b35-843f659daba2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:58.819964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9db17-2fc3-412d-8b35-843f659daba2 None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=dd1b7173843b4da48e69da8b36d59226) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:58.820283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9db17-2fc3-412d-8b35-843f659daba2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:58.821714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9db17-2fc3-412d-8b35-843f659daba2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:58.833087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 348/694] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:48:58 2022] GET /identity/v3/role_assignments?user.id=dd1b7173843b4da48e69da8b36d59226&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:58.834487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9db17-2fc3-412d-8b35-843f659daba2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:58.924230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:58.994242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 347/695] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:48:59.006621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9db17-2fc3-412d-8b35-843f659daba2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:59.041543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b945b-af4a-4ed5-94bc-684f78aea617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:59.042518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b945b-af4a-4ed5-94bc-684f78aea617 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:59.043101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b945b-af4a-4ed5-94bc-684f78aea617 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:59.043570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b945b-af4a-4ed5-94bc-684f78aea617 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:59.045275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b945b-af4a-4ed5-94bc-684f78aea617 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:59.045812 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b945b-af4a-4ed5-94bc-684f78aea617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:59.047466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b945b-af4a-4ed5-94bc-684f78aea617 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:59.052862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 349/696] 10.222.0.86 () {56 vars in 1124 bytes} [Mon Aug 29 18:48:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:59.058466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5971a2-35e6-4de3-99d2-04f5d6bcf023 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:59.154780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:59.172897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5971a2-35e6-4de3-99d2-04f5d6bcf023 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:59.209188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f3a72-f7b6-4a45-9204-ac9bbe5ee081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:59.209796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f3a72-f7b6-4a45-9204-ac9bbe5ee081 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:59.210073 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f3a72-f7b6-4a45-9204-ac9bbe5ee081 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:59.210344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f3a72-f7b6-4a45-9204-ac9bbe5ee081 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:59.210962 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f3a72-f7b6-4a45-9204-ac9bbe5ee081 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:59.211288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f3a72-f7b6-4a45-9204-ac9bbe5ee081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:59.212639 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f3a72-f7b6-4a45-9204-ac9bbe5ee081 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:59.220253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 348/697] 10.222.0.86 () {56 vars in 1124 bytes} [Mon Aug 29 18:48:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:59.307779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b945b-af4a-4ed5-94bc-684f78aea617 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:59.441371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b945b-af4a-4ed5-94bc-684f78aea617 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:59.459652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:59.478417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdf829-41ee-4a8a-aaf8-d2b128f4efd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:59.479208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf829-41ee-4a8a-aaf8-d2b128f4efd6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:59.479394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf829-41ee-4a8a-aaf8-d2b128f4efd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:59.479680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf829-41ee-4a8a-aaf8-d2b128f4efd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:59.490237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdf829-41ee-4a8a-aaf8-d2b128f4efd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1dcdf829-41ee-4a8a-aaf8-d2b128f4efd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:48:59.496297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:59.497321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:59.498173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 350/698] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:48:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:59.503425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f3a72-f7b6-4a45-9204-ac9bbe5ee081 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:59.649240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f3a72-f7b6-4a45-9204-ac9bbe5ee081 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:59.696789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ed190-254a-4223-bbac-4b5ec13dce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:59.697403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ed190-254a-4223-bbac-4b5ec13dce10 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:59.697680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ed190-254a-4223-bbac-4b5ec13dce10 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:59.697952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ed190-254a-4223-bbac-4b5ec13dce10 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:59.698611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ed190-254a-4223-bbac-4b5ec13dce10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:48:59.698911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ed190-254a-4223-bbac-4b5ec13dce10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:59.700815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ed190-254a-4223-bbac-4b5ec13dce10 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:59.713741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 349/699] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:48:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:48:59.724111 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdf829-41ee-4a8a-aaf8-d2b128f4efd6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:59.865121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdf829-41ee-4a8a-aaf8-d2b128f4efd6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:48:59.916095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb240ae-8d40-4259-901f-cbc22fa8a9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:48:59.916696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb240ae-8d40-4259-901f-cbc22fa8a9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:59.916975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb240ae-8d40-4259-901f-cbc22fa8a9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:59.917243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb240ae-8d40-4259-901f-cbc22fa8a9d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:59.929794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb240ae-8d40-4259-901f-cbc22fa8a9d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:48:59.931302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb240ae-8d40-4259-901f-cbc22fa8a9d4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3eb240ae-8d40-4259-901f-cbc22fa8a9d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:48:59.952321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:48:59.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:48:59.954248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:48:59.954248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:48:59.954248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:48:59.954248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 351/700] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:48:59.960080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ed190-254a-4223-bbac-4b5ec13dce10 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:48:59.964257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe1f7a-2919-4c38-8c59-72bba5d5eb0d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:59.964547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe1f7a-2919-4c38-8c59-72bba5d5eb0d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:59.964820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe1f7a-2919-4c38-8c59-72bba5d5eb0d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:59.966308 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 352/701] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:48:59.971529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf0c21-e689-44ad-990e-bdd7b0455ce3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:48:59.971812 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf0c21-e689-44ad-990e-bdd7b0455ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:48:59.972089 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf0c21-e689-44ad-990e-bdd7b0455ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:48:59.987335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:48:59.992744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-94cf0c21-e689-44ad-990e-bdd7b0455ce3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:48:59.996334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-94cf0c21-e689-44ad-990e-bdd7b0455ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:00.023470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:00.084043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 353/702] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:49:00.134871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ed190-254a-4223-bbac-4b5ec13dce10 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:00.171261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fea0fa-1c6e-4b67-8139-bc6a5aaa0ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:00.172115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fea0fa-1c6e-4b67-8139-bc6a5aaa0ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:00.172536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fea0fa-1c6e-4b67-8139-bc6a5aaa0ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:00.172937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fea0fa-1c6e-4b67-8139-bc6a5aaa0ad0 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:00.173696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fea0fa-1c6e-4b67-8139-bc6a5aaa0ad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:00.174132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fea0fa-1c6e-4b67-8139-bc6a5aaa0ad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:00.175800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fea0fa-1c6e-4b67-8139-bc6a5aaa0ad0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:00.190432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 350/703] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:00.195985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf0c21-e689-44ad-990e-bdd7b0455ce3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:00.376635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf0c21-e689-44ad-990e-bdd7b0455ce3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:00.439597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df480f83-c717-4073-a130-b83cb8c71a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:00.440533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df480f83-c717-4073-a130-b83cb8c71a79 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:00.440964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df480f83-c717-4073-a130-b83cb8c71a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:00.441381 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df480f83-c717-4073-a130-b83cb8c71a79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:00.456079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df480f83-c717-4073-a130-b83cb8c71a79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:00.457876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df480f83-c717-4073-a130-b83cb8c71a79 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-df480f83-c717-4073-a130-b83cb8c71a79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:00.479028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:00.480635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:00.482184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:00.482184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:00.482184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:00.482184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:00.482184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:00.482184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:00.482184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:00.482184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:00.482184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:00.482184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:00.484494 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 354/704] 10.222.0.86 () {56 vars in 1093 bytes} [Mon Aug 29 18:49:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:00.495592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fea0fa-1c6e-4b67-8139-bc6a5aaa0ad0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:00.511555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df480f83-c717-4073-a130-b83cb8c71a79 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:00.575417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:00.676674 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:00.710889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df480f83-c717-4073-a130-b83cb8c71a79 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:00.739300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fea0fa-1c6e-4b67-8139-bc6a5aaa0ad0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:00.771406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc7a0a-d376-4527-96c9-14153c3d3641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:00.772097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc7a0a-d376-4527-96c9-14153c3d3641 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:00.772322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc7a0a-d376-4527-96c9-14153c3d3641 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:00.772592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc7a0a-d376-4527-96c9-14153c3d3641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:00.778655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dc5c7-d0c2-41eb-9df8-bdcf1d9f949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:00.779302 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc5c7-d0c2-41eb-9df8-bdcf1d9f949f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:00.779577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc5c7-d0c2-41eb-9df8-bdcf1d9f949f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:00.779846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc5c7-d0c2-41eb-9df8-bdcf1d9f949f None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:00.780573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dc5c7-d0c2-41eb-9df8-bdcf1d9f949f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:00.780961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dc5c7-d0c2-41eb-9df8-bdcf1d9f949f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:00.787495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc7a0a-d376-4527-96c9-14153c3d3641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:00.788280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dc5c7-d0c2-41eb-9df8-bdcf1d9f949f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:00.789481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc7a0a-d376-4527-96c9-14153c3d3641 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d5cc7a0a-d376-4527-96c9-14153c3d3641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:00.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:00.809552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:00.810436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:00.810436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:00.810436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:00.817242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 355/705] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:00.820049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c0312-4e17-4a3b-9926-21453c446384 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:00.820567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c0312-4e17-4a3b-9926-21453c446384 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:00.821047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c0312-4e17-4a3b-9926-21453c446384 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:00.823224 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 356/706] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:00.824249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c0312-4e17-4a3b-9926-21453c446384 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:00.837062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 351/707] 10.222.0.86 () {56 vars in 1133 bytes} [Mon Aug 29 18:49:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:00.837062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd2c3d-b0fe-4d6f-8503-23a561c083ca None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:00.837062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd2c3d-b0fe-4d6f-8503-23a561c083ca None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:00.837062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd2c3d-b0fe-4d6f-8503-23a561c083ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:00.865629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-dfdd2c3d-b0fe-4d6f-8503-23a561c083ca None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:00.867779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dfdd2c3d-b0fe-4d6f-8503-23a561c083ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:00.956529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 352/708] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:49:00.957427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd2c3d-b0fe-4d6f-8503-23a561c083ca None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:01.078762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:01.095032 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:01.095886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd2c3d-b0fe-4d6f-8503-23a561c083ca None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:01.142499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c0312-4e17-4a3b-9926-21453c446384 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:01.162582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df1edb-ede1-44fd-8150-02cf74181374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:01.163218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1edb-ede1-44fd-8150-02cf74181374 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:01.163497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1edb-ede1-44fd-8150-02cf74181374 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:01.163769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1edb-ede1-44fd-8150-02cf74181374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:01.229248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573557-03c2-43c6-9718-0fafb5b913e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:01.230908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573557-03c2-43c6-9718-0fafb5b913e1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:01.231213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573557-03c2-43c6-9718-0fafb5b913e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:01.231520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573557-03c2-43c6-9718-0fafb5b913e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:01.232170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85573557-03c2-43c6-9718-0fafb5b913e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:01.232472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85573557-03c2-43c6-9718-0fafb5b913e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:01.233874 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85573557-03c2-43c6-9718-0fafb5b913e1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:01.257003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 357/709] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:01.266572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573557-03c2-43c6-9718-0fafb5b913e1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:01.284307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df1edb-ede1-44fd-8150-02cf74181374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:01.288501 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df1edb-ede1-44fd-8150-02cf74181374 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:01.358034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:01.398244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573557-03c2-43c6-9718-0fafb5b913e1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:01.407833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 353/710] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:49:01.408927 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df1edb-ede1-44fd-8150-02cf74181374 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:01.442200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:01.449055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd1896-043a-4f97-bcfa-e86638ea406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:01.450237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd1896-043a-4f97-bcfa-e86638ea406f None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:01.450518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd1896-043a-4f97-bcfa-e86638ea406f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:01.450787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd1896-043a-4f97-bcfa-e86638ea406f None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:01.451921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd1896-043a-4f97-bcfa-e86638ea406f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:01.453931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd1896-043a-4f97-bcfa-e86638ea406f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:01.487115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 358/711] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:49:01 2022] POST /identity/v3/users => generated 254 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:49:01.487115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52bc4f-6bcd-43ba-9b4c-f32499413245 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:01.487115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52bc4f-6bcd-43ba-9b4c-f32499413245 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:01.487115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52bc4f-6bcd-43ba-9b4c-f32499413245 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:01.491435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 359/712] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:01.492311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52bc4f-6bcd-43ba-9b4c-f32499413245 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:01.521462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df1edb-ede1-44fd-8150-02cf74181374 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:01.567050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b2b49-1322-4e73-8c51-2f73f78b7506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:01.567819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b2b49-1322-4e73-8c51-2f73f78b7506 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:01.568099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b2b49-1322-4e73-8c51-2f73f78b7506 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:01.568484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b2b49-1322-4e73-8c51-2f73f78b7506 None admin] PATH_INFO: `/v3/services/network` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:01.569329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b2b49-1322-4e73-8c51-2f73f78b7506 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:01.570721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b2b49-1322-4e73-8c51-2f73f78b7506 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-006b2b49-1322-4e73-8c51-2f73f78b7506 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:01.581328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:01.582484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:49:01.583417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:01.587396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 354/713] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:01 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:01.588898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666f0cf-0193-4c0a-87b0-87057efcde45 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:01.589187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666f0cf-0193-4c0a-87b0-87057efcde45 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:01.589467 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666f0cf-0193-4c0a-87b0-87057efcde45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:01.609749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f666f0cf-0193-4c0a-87b0-87057efcde45 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:01.611117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f666f0cf-0193-4c0a-87b0-87057efcde45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:01.631450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:01.634524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52bc4f-6bcd-43ba-9b4c-f32499413245 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:01.683069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc6a44-09cd-4196-9f5a-4b773182e42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:01.684043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc6a44-09cd-4196-9f5a-4b773182e42d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:01.684319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc6a44-09cd-4196-9f5a-4b773182e42d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:01.684625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc6a44-09cd-4196-9f5a-4b773182e42d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:01.697933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc6a44-09cd-4196-9f5a-4b773182e42d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:01.699690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc6a44-09cd-4196-9f5a-4b773182e42d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:01.702175 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 355/714] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:49:01.704283 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666f0cf-0193-4c0a-87b0-87057efcde45 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0ebc6a44-09cd-4196-9f5a-4b773182e42d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:01.721048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:01.722125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:01.723052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:01.723052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:01.723052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:01.725213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 360/715] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:01.737092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc6a44-09cd-4196-9f5a-4b773182e42d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:01.798429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:01.827634 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666f0cf-0193-4c0a-87b0-87057efcde45 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:01.870280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df147532-837b-4790-bcd3-8b8fbbfeda48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:01.871208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df147532-837b-4790-bcd3-8b8fbbfeda48 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:01.871487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df147532-837b-4790-bcd3-8b8fbbfeda48 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:01.871760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df147532-837b-4790-bcd3-8b8fbbfeda48 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:01.872948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df147532-837b-4790-bcd3-8b8fbbfeda48 None admin] RBAC: Adding query filter params (name=network) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:01.873254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df147532-837b-4790-bcd3-8b8fbbfeda48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:01.873619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc6a44-09cd-4196-9f5a-4b773182e42d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:01.874660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df147532-837b-4790-bcd3-8b8fbbfeda48 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:01.880806 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 356/716] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:01.882615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1568db-7303-47d6-8c5f-2023415cf903 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:01.882898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1568db-7303-47d6-8c5f-2023415cf903 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:01.883221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1568db-7303-47d6-8c5f-2023415cf903 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:01.885149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 357/717] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:01.886736 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1568db-7303-47d6-8c5f-2023415cf903 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:01.911491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:01.919739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba22bfb-4a9b-4f09-a88c-e1d0d4f033c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:01.920671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22bfb-4a9b-4f09-a88c-e1d0d4f033c5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:01.921520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22bfb-4a9b-4f09-a88c-e1d0d4f033c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:01.922427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22bfb-4a9b-4f09-a88c-e1d0d4f033c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:01.923275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba22bfb-4a9b-4f09-a88c-e1d0d4f033c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:01.923765 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba22bfb-4a9b-4f09-a88c-e1d0d4f033c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:01.925503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba22bfb-4a9b-4f09-a88c-e1d0d4f033c5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:01.939547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 361/718] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:01.946033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caefa5-4918-43be-930c-99bc72c4d564 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:01.946559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caefa5-4918-43be-930c-99bc72c4d564 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:01.947038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caefa5-4918-43be-930c-99bc72c4d564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:01.967935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-80caefa5-4918-43be-930c-99bc72c4d564 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:01.969684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-80caefa5-4918-43be-930c-99bc72c4d564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:02.002044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1568db-7303-47d6-8c5f-2023415cf903 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:02.037980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246c6f7-b0a4-462c-948f-056fc9d361a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:02.038593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246c6f7-b0a4-462c-948f-056fc9d361a8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:02.038882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246c6f7-b0a4-462c-948f-056fc9d361a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:02.039178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246c6f7-b0a4-462c-948f-056fc9d361a8 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:02.039801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246c6f7-b0a4-462c-948f-056fc9d361a8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:02.040226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246c6f7-b0a4-462c-948f-056fc9d361a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:02.041644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246c6f7-b0a4-462c-948f-056fc9d361a8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:02.043989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 362/719] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:49:02.045475 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80caefa5-4918-43be-930c-99bc72c4d564 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:02.050760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 358/720] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:02.051896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246c6f7-b0a4-462c-948f-056fc9d361a8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:02.121287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:02.190797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:02.225899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246c6f7-b0a4-462c-948f-056fc9d361a8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:02.243475 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80caefa5-4918-43be-930c-99bc72c4d564 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:02.267798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7131a-8a46-496b-93fb-234002be4b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:02.268518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7131a-8a46-496b-93fb-234002be4b91 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:02.268804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7131a-8a46-496b-93fb-234002be4b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:02.269079 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7131a-8a46-496b-93fb-234002be4b91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:02.286921 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7131a-8a46-496b-93fb-234002be4b91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:02.291111 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b132d-a3af-4159-9b74-b66a1590aeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:02.291745 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b132d-a3af-4159-9b74-b66a1590aeff None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:02.292021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b132d-a3af-4159-9b74-b66a1590aeff None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:02.292355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b132d-a3af-4159-9b74-b66a1590aeff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f5f7131a-8a46-496b-93fb-234002be4b91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:02.293012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:02.293957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:02.295225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 359/721] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:49:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:02.300279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7131a-8a46-496b-93fb-234002be4b91 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:02.312360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b132d-a3af-4159-9b74-b66a1590aeff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:02.313828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b132d-a3af-4159-9b74-b66a1590aeff None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a56b132d-a3af-4159-9b74-b66a1590aeff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:02.336409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:02.338453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:02.340759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:02.340759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:02.340759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:02.340759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:02.340759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:02.340759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:02.340759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:02.340759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:02.340759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 363/722] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:02.345450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b132d-a3af-4159-9b74-b66a1590aeff None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:02.444358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7131a-8a46-496b-93fb-234002be4b91 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:02.466649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:02.480199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b132d-a3af-4159-9b74-b66a1590aeff None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:02.485555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45295e-56ce-4644-b9bb-9814fc46d391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:02.486164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45295e-56ce-4644-b9bb-9814fc46d391 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:02.486442 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45295e-56ce-4644-b9bb-9814fc46d391 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:02.486714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45295e-56ce-4644-b9bb-9814fc46d391 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:02.487454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45295e-56ce-4644-b9bb-9814fc46d391 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:02.487752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45295e-56ce-4644-b9bb-9814fc46d391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:02.489258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45295e-56ce-4644-b9bb-9814fc46d391 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:02.499965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 360/723] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:49:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:02.501009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45295e-56ce-4644-b9bb-9814fc46d391 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:02.509242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:02.525904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e4a45-61a9-4e27-b8ac-c8faf8005ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:02.526565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e4a45-61a9-4e27-b8ac-c8faf8005ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:02.526768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e4a45-61a9-4e27-b8ac-c8faf8005ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:02.527129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e4a45-61a9-4e27-b8ac-c8faf8005ed8 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:02.527828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e4a45-61a9-4e27-b8ac-c8faf8005ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:02.528167 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e4a45-61a9-4e27-b8ac-c8faf8005ed8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:02.529654 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e4a45-61a9-4e27-b8ac-c8faf8005ed8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:02.547119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 364/724] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:02.548010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e4a45-61a9-4e27-b8ac-c8faf8005ed8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:02.631988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45295e-56ce-4644-b9bb-9814fc46d391 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:02.668543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb8ef0-2e4f-4d8a-8858-8956deee3cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:02.670041 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb8ef0-2e4f-4d8a-8858-8956deee3cba None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:02.670258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e4a45-61a9-4e27-b8ac-c8faf8005ed8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:02.670733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb8ef0-2e4f-4d8a-8858-8956deee3cba None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:02.671239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb8ef0-2e4f-4d8a-8858-8956deee3cba None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:02.672102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb8ef0-2e4f-4d8a-8858-8956deee3cba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:02.673676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb8ef0-2e4f-4d8a-8858-8956deee3cba None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:02.699327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 361/725] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:49:02 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:02.700567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb8ef0-2e4f-4d8a-8858-8956deee3cba None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:02.740477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01678a46-c474-4312-8091-832ae80a9977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:02.741137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01678a46-c474-4312-8091-832ae80a9977 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:02.741478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01678a46-c474-4312-8091-832ae80a9977 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:02.741751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01678a46-c474-4312-8091-832ae80a9977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:02.764490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01678a46-c474-4312-8091-832ae80a9977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:02.766052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01678a46-c474-4312-8091-832ae80a9977 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:02.775088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:02.788680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-01678a46-c474-4312-8091-832ae80a9977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:02.805045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:02.806096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:02.807652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:02.807652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:02.807652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:02.807652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 365/726] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:02.812250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01678a46-c474-4312-8091-832ae80a9977 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:02.829706 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb8ef0-2e4f-4d8a-8858-8956deee3cba None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:02.868479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecc0a0-9414-4dbb-b7bc-7f0162c94a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:02.869455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecc0a0-9414-4dbb-b7bc-7f0162c94a5d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:02.869957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecc0a0-9414-4dbb-b7bc-7f0162c94a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:02.870442 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecc0a0-9414-4dbb-b7bc-7f0162c94a5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:02.871554 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecc0a0-9414-4dbb-b7bc-7f0162c94a5d None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=045b22d046c549f2b53d2ed73b94f936) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:02.872182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecc0a0-9414-4dbb-b7bc-7f0162c94a5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:02.873809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecc0a0-9414-4dbb-b7bc-7f0162c94a5d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:02.880570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 362/727] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:02 2022] GET /identity/v3/role_assignments?user.id=045b22d046c549f2b53d2ed73b94f936&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:02.986208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01678a46-c474-4312-8091-832ae80a9977 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:03.030203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e96fa9-93f3-470f-b65a-05ff6c4240b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:03.031022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96fa9-93f3-470f-b65a-05ff6c4240b5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:03.031324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96fa9-93f3-470f-b65a-05ff6c4240b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:03.031601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96fa9-93f3-470f-b65a-05ff6c4240b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:03.032267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e96fa9-93f3-470f-b65a-05ff6c4240b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:03.032573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e96fa9-93f3-470f-b65a-05ff6c4240b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:03.033987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e96fa9-93f3-470f-b65a-05ff6c4240b5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:03.050299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 366/728] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:03.055919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecc0a0-9414-4dbb-b7bc-7f0162c94a5d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:03.170430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecc0a0-9414-4dbb-b7bc-7f0162c94a5d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:03.198689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:03.210310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd86f4-6449-4547-b2ae-5844860f371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:03.210947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd86f4-6449-4547-b2ae-5844860f371f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:03.211273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd86f4-6449-4547-b2ae-5844860f371f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:03.211550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd86f4-6449-4547-b2ae-5844860f371f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:03.220220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd86f4-6449-4547-b2ae-5844860f371f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:03.221661 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd86f4-6449-4547-b2ae-5844860f371f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5efd86f4-6449-4547-b2ae-5844860f371f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:03.233121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:03.234142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:03.235720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:03.235720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:03.235720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:03.235720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:03.235720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:03.235720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:03.235720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:03.235720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:03.235720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:49:03.235720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:03.235720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 363/729] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:49:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:03.240236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e96fa9-93f3-470f-b65a-05ff6c4240b5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:03.328575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:03.382829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e96fa9-93f3-470f-b65a-05ff6c4240b5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:03.427450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380021b-a269-4eae-8658-369238f7d215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:03.428452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380021b-a269-4eae-8658-369238f7d215 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:03.428964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380021b-a269-4eae-8658-369238f7d215 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:03.429432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380021b-a269-4eae-8658-369238f7d215 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:03.430356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380021b-a269-4eae-8658-369238f7d215 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:03.430857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380021b-a269-4eae-8658-369238f7d215 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:03.432509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380021b-a269-4eae-8658-369238f7d215 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:03.455609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 367/730] 10.222.0.86 () {56 vars in 1135 bytes} [Mon Aug 29 18:49:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:03.464085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd86f4-6449-4547-b2ae-5844860f371f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:03.601767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd86f4-6449-4547-b2ae-5844860f371f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:03.646428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640ae5b-b323-4bf2-b0aa-25a92149008e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:03.647509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640ae5b-b323-4bf2-b0aa-25a92149008e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:03.648038 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640ae5b-b323-4bf2-b0aa-25a92149008e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:03.648511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640ae5b-b323-4bf2-b0aa-25a92149008e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:03.716540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:03.729538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640ae5b-b323-4bf2-b0aa-25a92149008e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:03.736623 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640ae5b-b323-4bf2-b0aa-25a92149008e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:03.844840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 364/731] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:49:03.850629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4087b19-f636-43ba-a2dc-ce2268f52192 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:03.851114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4087b19-f636-43ba-a2dc-ce2268f52192 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:03.851223 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4087b19-f636-43ba-a2dc-ce2268f52192 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:03.852697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 368/732] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:03.853502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640ae5b-b323-4bf2-b0aa-25a92149008e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:03.858920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6361270-f98b-44c0-8aaf-bb8442eb8bb5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:03.859524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6361270-f98b-44c0-8aaf-bb8442eb8bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:03.860010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6361270-f98b-44c0-8aaf-bb8442eb8bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:03.880138 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e6361270-f98b-44c0-8aaf-bb8442eb8bb5 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:03.881749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e6361270-f98b-44c0-8aaf-bb8442eb8bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:03.973519 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:03.988459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640ae5b-b323-4bf2-b0aa-25a92149008e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:04.025159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76c33c-e5d8-426e-bad6-2d1711762ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:04.026163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76c33c-e5d8-426e-bad6-2d1711762ddd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:04.026659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76c33c-e5d8-426e-bad6-2d1711762ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:04.027183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76c33c-e5d8-426e-bad6-2d1711762ddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:04.029327 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 369/733] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:49:04.046754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76c33c-e5d8-426e-bad6-2d1711762ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:04.049270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76c33c-e5d8-426e-bad6-2d1711762ddd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5b76c33c-e5d8-426e-bad6-2d1711762ddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:04.067275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:04.068299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:04.069485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:04.069485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:04.069485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:04.072265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 365/734] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:04.080380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6361270-f98b-44c0-8aaf-bb8442eb8bb5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:04.151032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:04.227777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6361270-f98b-44c0-8aaf-bb8442eb8bb5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:04.269975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a49fe-8fd6-412c-9d88-af30865c4538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:04.270585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a49fe-8fd6-412c-9d88-af30865c4538 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:04.270860 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a49fe-8fd6-412c-9d88-af30865c4538 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:04.271151 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a49fe-8fd6-412c-9d88-af30865c4538 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:04.271790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a49fe-8fd6-412c-9d88-af30865c4538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:04.272106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a49fe-8fd6-412c-9d88-af30865c4538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:04.273499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a49fe-8fd6-412c-9d88-af30865c4538 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:04.286146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 370/735] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:04.287642 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58354f9c-90c6-4bb8-ab57-13d3f859bf11 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:04.288074 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58354f9c-90c6-4bb8-ab57-13d3f859bf11 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:04.288237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58354f9c-90c6-4bb8-ab57-13d3f859bf11 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:04.289763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 366/736] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:04.291110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a49fe-8fd6-412c-9d88-af30865c4538 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:04.295302 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012acf1-0395-45ef-aa4b-385bdd1d59a4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:04.295589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012acf1-0395-45ef-aa4b-385bdd1d59a4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:04.295865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012acf1-0395-45ef-aa4b-385bdd1d59a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:04.328397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9012acf1-0395-45ef-aa4b-385bdd1d59a4 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:04.329607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9012acf1-0395-45ef-aa4b-385bdd1d59a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:04.410127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a49fe-8fd6-412c-9d88-af30865c4538 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:04.412994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 367/737] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:49:04.414097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012acf1-0395-45ef-aa4b-385bdd1d59a4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:04.423931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:04.474604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bd10c-ea54-40e8-a029-5707073ec3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:04.475349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd10c-ea54-40e8-a029-5707073ec3ba None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:04.475633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd10c-ea54-40e8-a029-5707073ec3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:04.475904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd10c-ea54-40e8-a029-5707073ec3ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:04.486085 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bd10c-ea54-40e8-a029-5707073ec3ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:04.487589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bd10c-ea54-40e8-a029-5707073ec3ba None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:04.492268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0e2bd10c-ea54-40e8-a029-5707073ec3ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:04.507535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:04.508462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:04.509408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:04.509408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:04.509408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:04.509408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:04.509408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:04.509408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:04.509408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:04.509408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:04.509408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 371/738] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:04.519941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bd10c-ea54-40e8-a029-5707073ec3ba None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:04.534104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012acf1-0395-45ef-aa4b-385bdd1d59a4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:04.601706 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb34555-bf12-4459-94a5-4b942157eb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:04.602338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb34555-bf12-4459-94a5-4b942157eb24 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:04.602619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb34555-bf12-4459-94a5-4b942157eb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:04.602890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb34555-bf12-4459-94a5-4b942157eb24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:04.620039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb34555-bf12-4459-94a5-4b942157eb24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fcb34555-bf12-4459-94a5-4b942157eb24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:04.625417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:04.626376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:04.627270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 368/739] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:49:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:04.639987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb34555-bf12-4459-94a5-4b942157eb24 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:04.681104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bd10c-ea54-40e8-a029-5707073ec3ba None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:04.722749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1291a-2dde-4ba8-9fda-21f2212bc8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:04.723390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1291a-2dde-4ba8-9fda-21f2212bc8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:04.723670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1291a-2dde-4ba8-9fda-21f2212bc8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:04.723941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1291a-2dde-4ba8-9fda-21f2212bc8e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:04.724635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1291a-2dde-4ba8-9fda-21f2212bc8e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:04.724937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1291a-2dde-4ba8-9fda-21f2212bc8e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:04.726378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1291a-2dde-4ba8-9fda-21f2212bc8e7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:04.739666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 372/740] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:04.745417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1291a-2dde-4ba8-9fda-21f2212bc8e7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:04.774130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:04.836216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb34555-bf12-4459-94a5-4b942157eb24 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:04.858233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1291a-2dde-4ba8-9fda-21f2212bc8e7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:04.865739 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:04.872556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c80744-151f-4630-aeed-27a5c56bb829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:04.873940 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c80744-151f-4630-aeed-27a5c56bb829 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:04.874227 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c80744-151f-4630-aeed-27a5c56bb829 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:04.874505 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c80744-151f-4630-aeed-27a5c56bb829 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:04.875210 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c80744-151f-4630-aeed-27a5c56bb829 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:04.875518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c80744-151f-4630-aeed-27a5c56bb829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:04.876956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c80744-151f-4630-aeed-27a5c56bb829 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:04.885934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 369/741] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:49:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:04.886915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c80744-151f-4630-aeed-27a5c56bb829 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:04.899518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02b0b2-e108-4e48-ad6a-24401384517f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:04.900458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b0b2-e108-4e48-ad6a-24401384517f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:04.900956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b0b2-e108-4e48-ad6a-24401384517f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:04.901399 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b0b2-e108-4e48-ad6a-24401384517f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:04.902333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02b0b2-e108-4e48-ad6a-24401384517f None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=dd1b7173843b4da48e69da8b36d59226) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:04.902874 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02b0b2-e108-4e48-ad6a-24401384517f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:04.904616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02b0b2-e108-4e48-ad6a-24401384517f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:04.915171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 373/742] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:04 2022] GET /identity/v3/role_assignments?user.id=dd1b7173843b4da48e69da8b36d59226&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:04.916356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02b0b2-e108-4e48-ad6a-24401384517f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:05.003109 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c80744-151f-4630-aeed-27a5c56bb829 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:05.048794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b3ad4-2f9f-4ec9-9d2f-608dad1d8355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:05.049420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b3ad4-2f9f-4ec9-9d2f-608dad1d8355 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:05.049754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b3ad4-2f9f-4ec9-9d2f-608dad1d8355 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:05.050026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b3ad4-2f9f-4ec9-9d2f-608dad1d8355 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:05.050691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b3ad4-2f9f-4ec9-9d2f-608dad1d8355 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:05.052106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b3ad4-2f9f-4ec9-9d2f-608dad1d8355 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:05.084515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 370/743] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:49:04 2022] POST /identity/v3/services => generated 240 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:05.096304 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:05.109856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02b0b2-e108-4e48-ad6a-24401384517f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:05.165210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b738b-e2cb-4076-a11f-1640c4d881e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:05.165865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b738b-e2cb-4076-a11f-1640c4d881e6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:05.166144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b738b-e2cb-4076-a11f-1640c4d881e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:05.166416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b738b-e2cb-4076-a11f-1640c4d881e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:05.179517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b738b-e2cb-4076-a11f-1640c4d881e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:05.181143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b738b-e2cb-4076-a11f-1640c4d881e6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-bf3b738b-e2cb-4076-a11f-1640c4d881e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:05.199342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:05.200396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:05.201286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:05.201286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:05.201286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:05.201286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 374/744] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:05.206536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b3ad4-2f9f-4ec9-9d2f-608dad1d8355 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:05.269081 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:05.296636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0ef8c-72e3-421c-825f-b88d958e1110 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:05.296948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0ef8c-72e3-421c-825f-b88d958e1110 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:05.297198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0ef8c-72e3-421c-825f-b88d958e1110 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:05.298808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 375/745] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:05.303943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0288237-2326-4910-8e5b-f6182740ef7e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:05.304230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0288237-2326-4910-8e5b-f6182740ef7e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:05.304508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0288237-2326-4910-8e5b-f6182740ef7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:05.323930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b3ad4-2f9f-4ec9-9d2f-608dad1d8355 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:05.330417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f0288237-2326-4910-8e5b-f6182740ef7e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:05.331788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f0288237-2326-4910-8e5b-f6182740ef7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:05.364516 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc355c-caa2-4f65-895e-87ef99142d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:05.365245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc355c-caa2-4f65-895e-87ef99142d4b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:05.365528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc355c-caa2-4f65-895e-87ef99142d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:05.365800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc355c-caa2-4f65-895e-87ef99142d4b None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:05.366447 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc355c-caa2-4f65-895e-87ef99142d4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:05.366772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc355c-caa2-4f65-895e-87ef99142d4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:05.369604 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc355c-caa2-4f65-895e-87ef99142d4b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:05.383141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 371/746] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:05.391942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc355c-caa2-4f65-895e-87ef99142d4b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:05.427021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 376/747] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:05.509729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4b3cb-524a-45f5-853f-ddb03e91f263 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:05.510108 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4b3cb-524a-45f5-853f-ddb03e91f263 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:05.510459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4b3cb-524a-45f5-853f-ddb03e91f263 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:05.510657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc355c-caa2-4f65-895e-87ef99142d4b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:05.512213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 377/748] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:05.517806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e1ddf-fd42-49eb-8a55-403aa8590206 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:05.518172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e1ddf-fd42-49eb-8a55-403aa8590206 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:05.518518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e1ddf-fd42-49eb-8a55-403aa8590206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:05.526204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:05.537814 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-577e1ddf-fd42-49eb-8a55-403aa8590206 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:05.539146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-577e1ddf-fd42-49eb-8a55-403aa8590206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:05.554169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87ff37-b1c8-4b8a-8cf6-52ab365baaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:05.554916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87ff37-b1c8-4b8a-8cf6-52ab365baaba None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:05.555222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87ff37-b1c8-4b8a-8cf6-52ab365baaba None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:05.555493 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87ff37-b1c8-4b8a-8cf6-52ab365baaba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:05.565193 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87ff37-b1c8-4b8a-8cf6-52ab365baaba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:05.566766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87ff37-b1c8-4b8a-8cf6-52ab365baaba None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-7f87ff37-b1c8-4b8a-8cf6-52ab365baaba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:05.580356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:05.581585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:05.582546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:05.582546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:05.582546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:05.582546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:05.582546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:05.582546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:05.582546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:05.582546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:05.582546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:05.582546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:05.582546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 372/749] 10.222.0.86 () {56 vars in 1117 bytes} [Mon Aug 29 18:49:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:05.589367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87ff37-b1c8-4b8a-8cf6-52ab365baaba None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:05.611560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:05.617362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 378/750] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:05.753354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87ff37-b1c8-4b8a-8cf6-52ab365baaba None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:05.793672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ecfbe-01f4-4cc5-b030-31d17200ffd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:05.794263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ecfbe-01f4-4cc5-b030-31d17200ffd4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:05.794546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ecfbe-01f4-4cc5-b030-31d17200ffd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:05.794819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ecfbe-01f4-4cc5-b030-31d17200ffd4 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:05.795626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ecfbe-01f4-4cc5-b030-31d17200ffd4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:05.795931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ecfbe-01f4-4cc5-b030-31d17200ffd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:05.797344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ecfbe-01f4-4cc5-b030-31d17200ffd4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:05.816218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 373/751] 10.222.0.86 () {56 vars in 1149 bytes} [Mon Aug 29 18:49:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:05.824064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e1ddf-fd42-49eb-8a55-403aa8590206 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:05.907903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ecfbe-01f4-4cc5-b030-31d17200ffd4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:05.936486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e1ddf-fd42-49eb-8a55-403aa8590206 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:05.960782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:05.977311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6f356-6b0f-4ad6-9e24-2d5091990f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:05.978269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6f356-6b0f-4ad6-9e24-2d5091990f45 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:05.978693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6f356-6b0f-4ad6-9e24-2d5091990f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:05.979136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6f356-6b0f-4ad6-9e24-2d5091990f45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:06.034979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ecfbe-01f4-4cc5-b030-31d17200ffd4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:06.051411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:06.055589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6f356-6b0f-4ad6-9e24-2d5091990f45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:06.057228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6f356-6b0f-4ad6-9e24-2d5091990f45 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:06.074010 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6bf7-e8de-4eaf-8904-34b2bf6a73c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:06.074753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6bf7-e8de-4eaf-8904-34b2bf6a73c0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:06.075622 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6bf7-e8de-4eaf-8904-34b2bf6a73c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:06.075901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6bf7-e8de-4eaf-8904-34b2bf6a73c0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:06.076488 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e6bf7-e8de-4eaf-8904-34b2bf6a73c0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:06.077851 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e6bf7-e8de-4eaf-8904-34b2bf6a73c0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-804e6bf7-e8de-4eaf-8904-34b2bf6a73c0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:06.087481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:06.088517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:06.089688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 374/752] 10.222.0.86 () {56 vars in 1129 bytes} [Mon Aug 29 18:49:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:06.090531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6bf7-e8de-4eaf-8904-34b2bf6a73c0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:06.167882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 379/753] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:49:06.169848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6f356-6b0f-4ad6-9e24-2d5091990f45 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:06.190692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:06.197748 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6bf7-e8de-4eaf-8904-34b2bf6a73c0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:06.240760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2c0fd-640a-4fc4-bcef-43facee99b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:06.241630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2c0fd-640a-4fc4-bcef-43facee99b40 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:06.242088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2c0fd-640a-4fc4-bcef-43facee99b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:06.242517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2c0fd-640a-4fc4-bcef-43facee99b40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:06.254148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2c0fd-640a-4fc4-bcef-43facee99b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8de2c0fd-640a-4fc4-bcef-43facee99b40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:06.259575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:06.260675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:06.261694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:06.264030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 375/754] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:49:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:06.265069 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2c0fd-640a-4fc4-bcef-43facee99b40 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:06.297393 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:06.302601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6f356-6b0f-4ad6-9e24-2d5091990f45 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:06.351287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd16741-f20d-4223-a72b-c174e69becb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:06.352045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16741-f20d-4223-a72b-c174e69becb0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:06.352660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16741-f20d-4223-a72b-c174e69becb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:06.353027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16741-f20d-4223-a72b-c174e69becb0 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:06.354345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd16741-f20d-4223-a72b-c174e69becb0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:06.354748 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd16741-f20d-4223-a72b-c174e69becb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:06.357028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd16741-f20d-4223-a72b-c174e69becb0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:06.371147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 380/755] 10.222.0.86 () {56 vars in 1124 bytes} [Mon Aug 29 18:49:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:06.372912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd16741-f20d-4223-a72b-c174e69becb0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:06.397794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2c0fd-640a-4fc4-bcef-43facee99b40 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:06.438311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b5b16-0a5a-43d9-97e8-bc17d6d05c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:06.439914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b5b16-0a5a-43d9-97e8-bc17d6d05c8f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:06.440419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b5b16-0a5a-43d9-97e8-bc17d6d05c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:06.441180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b5b16-0a5a-43d9-97e8-bc17d6d05c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:06.456797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b5b16-0a5a-43d9-97e8-bc17d6d05c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:06.459469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b5b16-0a5a-43d9-97e8-bc17d6d05c8f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e37b5b16-0a5a-43d9-97e8-bc17d6d05c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:06.479933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:06.480928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:06.482452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:06.482452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:06.482452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:06.484503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 376/756] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:06.485691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b5b16-0a5a-43d9-97e8-bc17d6d05c8f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:06.497220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:06.502622 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd16741-f20d-4223-a72b-c174e69becb0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:06.546742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc287957-c321-4533-87ca-02bd2d21af76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:06.547594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc287957-c321-4533-87ca-02bd2d21af76 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:06.547984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc287957-c321-4533-87ca-02bd2d21af76 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:06.548344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc287957-c321-4533-87ca-02bd2d21af76 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:06.549093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc287957-c321-4533-87ca-02bd2d21af76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:06.549591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc287957-c321-4533-87ca-02bd2d21af76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:06.551236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc287957-c321-4533-87ca-02bd2d21af76 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:06.560565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 381/757] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:49:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:06.561604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc287957-c321-4533-87ca-02bd2d21af76 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:06.591157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:06.618976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b5b16-0a5a-43d9-97e8-bc17d6d05c8f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:06.665483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe2ab0-81f1-4077-a425-2f31a50a99dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:06.666858 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe2ab0-81f1-4077-a425-2f31a50a99dd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:06.667370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe2ab0-81f1-4077-a425-2f31a50a99dd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:06.667675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe2ab0-81f1-4077-a425-2f31a50a99dd None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:06.668280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe2ab0-81f1-4077-a425-2f31a50a99dd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:06.668584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe2ab0-81f1-4077-a425-2f31a50a99dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:06.670489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe2ab0-81f1-4077-a425-2f31a50a99dd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:06.676080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 377/758] 10.222.0.86 () {56 vars in 1124 bytes} [Mon Aug 29 18:49:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:06.676878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe2ab0-81f1-4077-a425-2f31a50a99dd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:06.681390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc287957-c321-4533-87ca-02bd2d21af76 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:06.728079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d784774-6f9c-4858-aea8-dd9b0d6a1f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:06.728939 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d784774-6f9c-4858-aea8-dd9b0d6a1f36 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:06.729522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d784774-6f9c-4858-aea8-dd9b0d6a1f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:06.729945 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d784774-6f9c-4858-aea8-dd9b0d6a1f36 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:06.731045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d784774-6f9c-4858-aea8-dd9b0d6a1f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:06.732095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d784774-6f9c-4858-aea8-dd9b0d6a1f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:06.734145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d784774-6f9c-4858-aea8-dd9b0d6a1f36 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:06.754679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:06.755101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 382/759] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:06.756157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d784774-6f9c-4858-aea8-dd9b0d6a1f36 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:06.789144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe2ab0-81f1-4077-a425-2f31a50a99dd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:06.827226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16b4ea-1865-4fe0-8024-28840a77a52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:06.827960 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16b4ea-1865-4fe0-8024-28840a77a52b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:06.828331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16b4ea-1865-4fe0-8024-28840a77a52b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:06.828697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16b4ea-1865-4fe0-8024-28840a77a52b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:06.840077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16b4ea-1865-4fe0-8024-28840a77a52b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:06.844570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16b4ea-1865-4fe0-8024-28840a77a52b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1f16b4ea-1865-4fe0-8024-28840a77a52b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:06.860710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:06.861917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:06.863067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:06.863067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:06.863067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:06.864450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 378/760] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:06.865445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16b4ea-1865-4fe0-8024-28840a77a52b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:06.866795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:06.888891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d784774-6f9c-4858-aea8-dd9b0d6a1f36 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:06.935030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007ac6f-9e4a-4886-935a-3d62caffe32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:06.935891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007ac6f-9e4a-4886-935a-3d62caffe32b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:06.936321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007ac6f-9e4a-4886-935a-3d62caffe32b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:06.936867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007ac6f-9e4a-4886-935a-3d62caffe32b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:06.938749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007ac6f-9e4a-4886-935a-3d62caffe32b None admin] RBAC: Adding query filter params (interface=public, service_id=147e1908e25d436fac875a090792f52c, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:06.939278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007ac6f-9e4a-4886-935a-3d62caffe32b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:06.940859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007ac6f-9e4a-4886-935a-3d62caffe32b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:06.952514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 383/761] 10.222.0.86 () {56 vars in 1251 bytes} [Mon Aug 29 18:49:06 2022] GET /identity/v3/endpoints?service_id=147e1908e25d436fac875a090792f52c&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:06.953540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007ac6f-9e4a-4886-935a-3d62caffe32b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:06.990815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16b4ea-1865-4fe0-8024-28840a77a52b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:07.022812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:07.038790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd7bb4-eb2d-4cc2-aa7d-16abe1a83d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:07.039681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd7bb4-eb2d-4cc2-aa7d-16abe1a83d1f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.040111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd7bb4-eb2d-4cc2-aa7d-16abe1a83d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.040996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd7bb4-eb2d-4cc2-aa7d-16abe1a83d1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.053473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd7bb4-eb2d-4cc2-aa7d-16abe1a83d1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:07.055244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd7bb4-eb2d-4cc2-aa7d-16abe1a83d1f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:07.069283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007ac6f-9e4a-4886-935a-3d62caffe32b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1fcd7bb4-eb2d-4cc2-aa7d-16abe1a83d1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:07.073936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:07.075593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:07.076708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:07.076708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:07.076708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:07.076708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:07.076708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:07.076708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:07.076708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:07.076708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:07.083700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 379/762] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:07.083939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd7bb4-eb2d-4cc2-aa7d-16abe1a83d1f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:07.123167 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930f368-03e2-4278-9980-8635eb00e1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:07.124925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930f368-03e2-4278-9980-8635eb00e1ec None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.125394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930f368-03e2-4278-9980-8635eb00e1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.125898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930f368-03e2-4278-9980-8635eb00e1ec None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.126757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930f368-03e2-4278-9980-8635eb00e1ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:07.127262 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930f368-03e2-4278-9980-8635eb00e1ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:07.129264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930f368-03e2-4278-9980-8635eb00e1ec None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:07.145947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 384/763] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:07.147683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930f368-03e2-4278-9980-8635eb00e1ec None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:07.161301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:07.196367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd7bb4-eb2d-4cc2-aa7d-16abe1a83d1f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:07.232618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd65fad-dcec-4ed4-bf2a-4774cb9014cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:07.233519 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd65fad-dcec-4ed4-bf2a-4774cb9014cf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.234361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd65fad-dcec-4ed4-bf2a-4774cb9014cf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.234810 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd65fad-dcec-4ed4-bf2a-4774cb9014cf None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.235621 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd65fad-dcec-4ed4-bf2a-4774cb9014cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:07.237760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd65fad-dcec-4ed4-bf2a-4774cb9014cf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:07.252044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 380/764] 10.222.0.86 () {56 vars in 1087 bytes} [Mon Aug 29 18:49:07 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:07.253538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd65fad-dcec-4ed4-bf2a-4774cb9014cf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:07.262117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930f368-03e2-4278-9980-8635eb00e1ec None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:07.303025 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a49fd9-773a-432c-adeb-a14d4332c660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:07.303844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a49fd9-773a-432c-adeb-a14d4332c660 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.304125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a49fd9-773a-432c-adeb-a14d4332c660 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.304392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a49fd9-773a-432c-adeb-a14d4332c660 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.305091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a49fd9-773a-432c-adeb-a14d4332c660 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:07.305392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a49fd9-773a-432c-adeb-a14d4332c660 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:07.306820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a49fd9-773a-432c-adeb-a14d4332c660 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:07.321287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:07.321496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 385/765] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:07.323158 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae4380-d103-4f27-b22b-68e9e92bac44 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.323485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae4380-d103-4f27-b22b-68e9e92bac44 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.323714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae4380-d103-4f27-b22b-68e9e92bac44 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.329567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 386/766] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:07.329567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae4380-d103-4f27-b22b-68e9e92bac44 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:07.390466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd65fad-dcec-4ed4-bf2a-4774cb9014cf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:07.421920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:07.438248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c20b5-b34a-4924-9604-a5d212c5d647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:07.439305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c20b5-b34a-4924-9604-a5d212c5d647 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.439601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c20b5-b34a-4924-9604-a5d212c5d647 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.439871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c20b5-b34a-4924-9604-a5d212c5d647 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.448863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c20b5-b34a-4924-9604-a5d212c5d647 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:07.450879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c20b5-b34a-4924-9604-a5d212c5d647 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:07.458735 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae4380-d103-4f27-b22b-68e9e92bac44 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-024c20b5-b34a-4924-9604-a5d212c5d647 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:07.463300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:07.464223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:07.465151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:07.465151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:07.465151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:07.465151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:07.465151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:07.465151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:07.465151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:07.465151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:07.465151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:07.465151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:07.465151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 381/767] 10.222.0.86 () {56 vars in 1093 bytes} [Mon Aug 29 18:49:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:07.466286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914b607-3596-43ab-ae0f-2c6a07636693 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.466286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914b607-3596-43ab-ae0f-2c6a07636693 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.466440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914b607-3596-43ab-ae0f-2c6a07636693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.487033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7914b607-3596-43ab-ae0f-2c6a07636693 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:07.488220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7914b607-3596-43ab-ae0f-2c6a07636693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:07.501716 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea4a11-fa66-454d-af9e-c53aa622607a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:07.502324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea4a11-fa66-454d-af9e-c53aa622607a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.502602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea4a11-fa66-454d-af9e-c53aa622607a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.502877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea4a11-fa66-454d-af9e-c53aa622607a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.503630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea4a11-fa66-454d-af9e-c53aa622607a None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=7634f750a07143a8b8d91cf0c8e78e16) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:07.503948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea4a11-fa66-454d-af9e-c53aa622607a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:07.505375 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea4a11-fa66-454d-af9e-c53aa622607a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:07.519027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 387/768] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:07 2022] GET /identity/v3/role_assignments?user.id=7634f750a07143a8b8d91cf0c8e78e16&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:07.519027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d724ca3-a621-4d59-bf69-ed982b3850de None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.519027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d724ca3-a621-4d59-bf69-ed982b3850de None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.519027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d724ca3-a621-4d59-bf69-ed982b3850de None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.519027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 388/769] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:07.519027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d724ca3-a621-4d59-bf69-ed982b3850de None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:07.601200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 382/770] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:07.602771 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28a6ba-c153-4722-929b-b9fe4c218dd5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.603089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28a6ba-c153-4722-929b-b9fe4c218dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.603487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28a6ba-c153-4722-929b-b9fe4c218dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.640114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5b28a6ba-c153-4722-929b-b9fe4c218dd5 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:07.642408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5b28a6ba-c153-4722-929b-b9fe4c218dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:07.643302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d724ca3-a621-4d59-bf69-ed982b3850de None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:07.698245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:07.701969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:07.714378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6592c3f-0a79-4752-afaa-e4c33cd63514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:07.715767 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6592c3f-0a79-4752-afaa-e4c33cd63514 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.717244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6592c3f-0a79-4752-afaa-e4c33cd63514 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.717632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6592c3f-0a79-4752-afaa-e4c33cd63514 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.718398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6592c3f-0a79-4752-afaa-e4c33cd63514 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:07.719276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6592c3f-0a79-4752-afaa-e4c33cd63514 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:07.721509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6592c3f-0a79-4752-afaa-e4c33cd63514 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:07.765388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 389/771] 10.222.0.86 () {56 vars in 1133 bytes} [Mon Aug 29 18:49:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:07.766288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 383/772] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:07.771612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6592c3f-0a79-4752-afaa-e4c33cd63514 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:07.879464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6592c3f-0a79-4752-afaa-e4c33cd63514 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:07.920497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b681f2f-42d5-4e8c-a666-56d5798be458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:07.921489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b681f2f-42d5-4e8c-a666-56d5798be458 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:07.921984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b681f2f-42d5-4e8c-a666-56d5798be458 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:07.922450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b681f2f-42d5-4e8c-a666-56d5798be458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:07.933464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b681f2f-42d5-4e8c-a666-56d5798be458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:07.936033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b681f2f-42d5-4e8c-a666-56d5798be458 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:07.948147 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28a6ba-c153-4722-929b-b9fe4c218dd5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9b681f2f-42d5-4e8c-a666-56d5798be458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:07.953845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:07.955110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:07.956029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:07.956029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:07.956029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:07.958027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 390/773] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:07.964677 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b681f2f-42d5-4e8c-a666-56d5798be458 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:08.019180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:08.091085 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b681f2f-42d5-4e8c-a666-56d5798be458 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:08.131343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be85d96-033d-44a4-ba35-7aaf8f92d0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:08.132619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be85d96-033d-44a4-ba35-7aaf8f92d0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:08.133125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be85d96-033d-44a4-ba35-7aaf8f92d0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:08.133619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be85d96-033d-44a4-ba35-7aaf8f92d0d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:08.134478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be85d96-033d-44a4-ba35-7aaf8f92d0d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:08.134978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be85d96-033d-44a4-ba35-7aaf8f92d0d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:08.136855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be85d96-033d-44a4-ba35-7aaf8f92d0d4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:08.155185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 391/774] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:08.156961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be85d96-033d-44a4-ba35-7aaf8f92d0d4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:08.198673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28a6ba-c153-4722-929b-b9fe4c218dd5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:08.237765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb0402-0653-4885-98bb-30a5d9f6687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:08.238759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb0402-0653-4885-98bb-30a5d9f6687c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:08.239365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb0402-0653-4885-98bb-30a5d9f6687c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:08.239837 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb0402-0653-4885-98bb-30a5d9f6687c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:08.252591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb0402-0653-4885-98bb-30a5d9f6687c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1eeb0402-0653-4885-98bb-30a5d9f6687c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:08.258078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:08.259023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:08.259871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:08.261264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be85d96-033d-44a4-ba35-7aaf8f92d0d4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:08.262793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 384/775] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:49:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:08.263372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:08.265210 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb0402-0653-4885-98bb-30a5d9f6687c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:08.299262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:08.299566 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b13d28-81d0-4626-9a29-29fd1dc621a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:08.300631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b13d28-81d0-4626-9a29-29fd1dc621a0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:08.300908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b13d28-81d0-4626-9a29-29fd1dc621a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:08.301189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b13d28-81d0-4626-9a29-29fd1dc621a0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:08.302028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b13d28-81d0-4626-9a29-29fd1dc621a0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:08.303410 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b13d28-81d0-4626-9a29-29fd1dc621a0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d7b13d28-81d0-4626-9a29-29fd1dc621a0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:08.314982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:08.316070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:49:08.316921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:08.321364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 392/776] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:08 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:08.322271 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b13d28-81d0-4626-9a29-29fd1dc621a0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:08.388217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb0402-0653-4885-98bb-30a5d9f6687c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:08.428256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734da09-ec8b-46d5-9f10-9a5c5d0aa99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:08.428941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b13d28-81d0-4626-9a29-29fd1dc621a0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:08.429616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734da09-ec8b-46d5-9f10-9a5c5d0aa99e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:08.430102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734da09-ec8b-46d5-9f10-9a5c5d0aa99e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:08.430692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734da09-ec8b-46d5-9f10-9a5c5d0aa99e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:08.441475 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734da09-ec8b-46d5-9f10-9a5c5d0aa99e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:08.444102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734da09-ec8b-46d5-9f10-9a5c5d0aa99e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2734da09-ec8b-46d5-9f10-9a5c5d0aa99e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:08.458258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:08.459483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:08.461195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:08.461195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:08.461195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:08.461195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:08.461195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:08.461195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:08.461195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:08.461195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:08.468616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 385/777] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:08.470092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734da09-ec8b-46d5-9f10-9a5c5d0aa99e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:08.472254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9eae6-7f0f-4cf6-8668-6abd73e36658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:08.472880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9eae6-7f0f-4cf6-8668-6abd73e36658 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:08.473218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9eae6-7f0f-4cf6-8668-6abd73e36658 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:08.473493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9eae6-7f0f-4cf6-8668-6abd73e36658 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:08.474222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9eae6-7f0f-4cf6-8668-6abd73e36658 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:08.474527 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9eae6-7f0f-4cf6-8668-6abd73e36658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:08.476580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9eae6-7f0f-4cf6-8668-6abd73e36658 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:08.485218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 393/778] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:49:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:08.486046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9eae6-7f0f-4cf6-8668-6abd73e36658 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:08.516215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:08.570492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:08.589457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734da09-ec8b-46d5-9f10-9a5c5d0aa99e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:08.591785 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9eae6-7f0f-4cf6-8668-6abd73e36658 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:08.626485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a338688-c730-49c2-ad9b-d237501bc796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:08.627155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338688-c730-49c2-ad9b-d237501bc796 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:08.627509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338688-c730-49c2-ad9b-d237501bc796 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:08.627789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338688-c730-49c2-ad9b-d237501bc796 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:08.628487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a338688-c730-49c2-ad9b-d237501bc796 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:08.628684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2723e77-b48a-45fa-9390-b3fcc07f36aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:08.628802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a338688-c730-49c2-ad9b-d237501bc796 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:08.629243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2723e77-b48a-45fa-9390-b3fcc07f36aa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:08.629529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2723e77-b48a-45fa-9390-b3fcc07f36aa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:08.629802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2723e77-b48a-45fa-9390-b3fcc07f36aa None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:08.630218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a338688-c730-49c2-ad9b-d237501bc796 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:08.630427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2723e77-b48a-45fa-9390-b3fcc07f36aa None admin] RBAC: Adding query filter params (name=network) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:08.630725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2723e77-b48a-45fa-9390-b3fcc07f36aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:08.632219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2723e77-b48a-45fa-9390-b3fcc07f36aa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:08.637932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 386/779] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:08.638818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2723e77-b48a-45fa-9390-b3fcc07f36aa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:08.655082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 394/780] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:08.656095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a338688-c730-49c2-ad9b-d237501bc796 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:08.761517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:08.775239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a338688-c730-49c2-ad9b-d237501bc796 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:08.781377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2723e77-b48a-45fa-9390-b3fcc07f36aa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:08.811173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075722c-a207-4981-bc0b-ad18d976386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:08.811902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075722c-a207-4981-bc0b-ad18d976386b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:08.812188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075722c-a207-4981-bc0b-ad18d976386b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:08.812729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075722c-a207-4981-bc0b-ad18d976386b None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:08.813103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075722c-a207-4981-bc0b-ad18d976386b None admin] RBAC: Adding query filter params (type=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:08.813412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075722c-a207-4981-bc0b-ad18d976386b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:08.814852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075722c-a207-4981-bc0b-ad18d976386b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:08.821774 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1d960-4417-4430-956e-64a6fa894cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:08.822629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1d960-4417-4430-956e-64a6fa894cad None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:08.823083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1d960-4417-4430-956e-64a6fa894cad None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:08.823496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1d960-4417-4430-956e-64a6fa894cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:08.826946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 395/781] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:08.828000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075722c-a207-4981-bc0b-ad18d976386b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:08.836591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1d960-4417-4430-956e-64a6fa894cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:08.838358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1d960-4417-4430-956e-64a6fa894cad None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8ca1d960-4417-4430-956e-64a6fa894cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:08.854699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:08.855983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:08.857045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:08.857045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:08.857045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:08.859532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 387/782] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:08.860765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1d960-4417-4430-956e-64a6fa894cad None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:08.863925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:08.933519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075722c-a207-4981-bc0b-ad18d976386b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:08.969411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccc783-2025-4197-bd48-385dc4df413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:08.970070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccc783-2025-4197-bd48-385dc4df413b None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:08.970346 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccc783-2025-4197-bd48-385dc4df413b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:08.970624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccc783-2025-4197-bd48-385dc4df413b None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/045b22d046c549f2b53d2ed73b94f936/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:08.972725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ccc783-2025-4197-bd48-385dc4df413b None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=045b22d046c549f2b53d2ed73b94f936, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:08.974161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ccc783-2025-4197-bd48-385dc4df413b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:08.975987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1d960-4417-4430-956e-64a6fa894cad None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:08.995954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 396/783] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:49:08 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/045b22d046c549f2b53d2ed73b94f936/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:49:08.995954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccc783-2025-4197-bd48-385dc4df413b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:09.015591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:09.038851 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eda977-cb21-44e7-b587-3da9900103df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:09.039659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eda977-cb21-44e7-b587-3da9900103df None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:09.040007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eda977-cb21-44e7-b587-3da9900103df None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:09.040762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eda977-cb21-44e7-b587-3da9900103df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:09.041719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eda977-cb21-44e7-b587-3da9900103df None admin] RBAC: Adding query filter params (interface=public, service_id=16325b4b489e4407bd9180241001407f, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:09.042088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eda977-cb21-44e7-b587-3da9900103df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:09.051226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eda977-cb21-44e7-b587-3da9900103df None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:09.065716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 388/784] 10.222.0.86 () {56 vars in 1251 bytes} [Mon Aug 29 18:49:08 2022] GET /identity/v3/endpoints?service_id=16325b4b489e4407bd9180241001407f&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:09.071783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eda977-cb21-44e7-b587-3da9900103df None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:09.119649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccc783-2025-4197-bd48-385dc4df413b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:09.158681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4494a75-9b1f-4943-bcd8-d16ef797101f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:09.159381 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4494a75-9b1f-4943-bcd8-d16ef797101f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:09.159669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4494a75-9b1f-4943-bcd8-d16ef797101f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:09.159949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4494a75-9b1f-4943-bcd8-d16ef797101f None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:09.160643 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4494a75-9b1f-4943-bcd8-d16ef797101f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:09.160956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4494a75-9b1f-4943-bcd8-d16ef797101f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:09.162459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4494a75-9b1f-4943-bcd8-d16ef797101f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:09.177301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 397/785] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:09.185748 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4494a75-9b1f-4943-bcd8-d16ef797101f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:09.206914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:09.255828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eda977-cb21-44e7-b587-3da9900103df None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:09.300242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959ef42-ffe7-4a58-8bb3-7e49207b8f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:09.300777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959ef42-ffe7-4a58-8bb3-7e49207b8f9b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:09.301054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959ef42-ffe7-4a58-8bb3-7e49207b8f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:09.301323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959ef42-ffe7-4a58-8bb3-7e49207b8f9b None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:09.301542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:09.302248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959ef42-ffe7-4a58-8bb3-7e49207b8f9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:09.303318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959ef42-ffe7-4a58-8bb3-7e49207b8f9b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:09.309096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 389/786] 10.222.0.86 () {56 vars in 1087 bytes} [Mon Aug 29 18:49:09 2022] GET /identity/v3/services => generated 772 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:09.337830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2cd9e-6e76-454d-9813-4d53eeb648b3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:09.338151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2cd9e-6e76-454d-9813-4d53eeb648b3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:09.338540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2cd9e-6e76-454d-9813-4d53eeb648b3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:09.340450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 390/787] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:09.340700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4494a75-9b1f-4943-bcd8-d16ef797101f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:09.346037 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed12833-2527-487d-be05-c9c1dadc7805 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:09.346336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed12833-2527-487d-be05-c9c1dadc7805 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:09.346586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed12833-2527-487d-be05-c9c1dadc7805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:09.366582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bed12833-2527-487d-be05-c9c1dadc7805 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:09.367840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bed12833-2527-487d-be05-c9c1dadc7805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:09.390804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09168a16-b950-4022-be1e-753fd1de234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:09.391484 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168a16-b950-4022-be1e-753fd1de234d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:09.391762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168a16-b950-4022-be1e-753fd1de234d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:09.392036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168a16-b950-4022-be1e-753fd1de234d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:09.402722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09168a16-b950-4022-be1e-753fd1de234d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:09.404302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09168a16-b950-4022-be1e-753fd1de234d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-09168a16-b950-4022-be1e-753fd1de234d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:09.432492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:09.433535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:09.435621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:09.435621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:09.435621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:09.435621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:09.435621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:09.435621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:09.435621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:09.435621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:09.435621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:49:09.435621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:09.435621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 398/788] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:49:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:09.442839 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09168a16-b950-4022-be1e-753fd1de234d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:09.450387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 391/789] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:09.567322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09168a16-b950-4022-be1e-753fd1de234d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:09.600289 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:09.610008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbda4c-4982-4db7-b9cf-2ecfe626fc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:09.610636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbda4c-4982-4db7-b9cf-2ecfe626fc3e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:09.610918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbda4c-4982-4db7-b9cf-2ecfe626fc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:09.611220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbda4c-4982-4db7-b9cf-2ecfe626fc3e None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:09.611967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbda4c-4982-4db7-b9cf-2ecfe626fc3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:09.612274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbda4c-4982-4db7-b9cf-2ecfe626fc3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:09.613715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbda4c-4982-4db7-b9cf-2ecfe626fc3e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:09.633480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 399/790] 10.222.0.86 () {56 vars in 1135 bytes} [Mon Aug 29 18:49:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:09.639027 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed12833-2527-487d-be05-c9c1dadc7805 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:09.727431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:09.730389 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815148f-c700-46e1-b5ee-3588d9599203 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:09.730883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815148f-c700-46e1-b5ee-3588d9599203 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:09.731936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815148f-c700-46e1-b5ee-3588d9599203 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:09.733799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 400/791] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:09.739913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0415-30cc-4f11-abfc-95a518340570 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:09.740482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0415-30cc-4f11-abfc-95a518340570 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:09.740961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0415-30cc-4f11-abfc-95a518340570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:09.760715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-961d0415-30cc-4f11-abfc-95a518340570 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:09.762297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-961d0415-30cc-4f11-abfc-95a518340570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:09.804477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed12833-2527-487d-be05-c9c1dadc7805 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:09.849586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d20b5-7a87-43dc-88b5-ac88b32b9be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:09.850173 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d20b5-7a87-43dc-88b5-ac88b32b9be6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:09.850449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d20b5-7a87-43dc-88b5-ac88b32b9be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:09.850718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d20b5-7a87-43dc-88b5-ac88b32b9be6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:09.852323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 401/792] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:09.854070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d0415-30cc-4f11-abfc-95a518340570 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:09.863033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d20b5-7a87-43dc-88b5-ac88b32b9be6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:09.865003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d20b5-7a87-43dc-88b5-ac88b32b9be6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-784d20b5-7a87-43dc-88b5-ac88b32b9be6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:09.891023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:09.892040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:09.892946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:09.892946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:09.892946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:09.892946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 392/793] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:09.892946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d20b5-7a87-43dc-88b5-ac88b32b9be6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:10.042932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d20b5-7a87-43dc-88b5-ac88b32b9be6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:10.058395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d0415-30cc-4f11-abfc-95a518340570 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:10.074867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:10.098656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:10.114232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b42ae8-d7ca-4ce8-aac6-7f1b362f5711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:10.114771 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b42ae8-d7ca-4ce8-aac6-7f1b362f5711 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:10.115071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b42ae8-d7ca-4ce8-aac6-7f1b362f5711 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:10.115342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b42ae8-d7ca-4ce8-aac6-7f1b362f5711 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:10.115982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b42ae8-d7ca-4ce8-aac6-7f1b362f5711 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:10.116282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b42ae8-d7ca-4ce8-aac6-7f1b362f5711 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:10.117638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b42ae8-d7ca-4ce8-aac6-7f1b362f5711 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:10.130965 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038c07b-86a8-4ded-9de5-d9ee06d0ceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:10.131749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038c07b-86a8-4ded-9de5-d9ee06d0ceb2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:10.132032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038c07b-86a8-4ded-9de5-d9ee06d0ceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:10.132303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038c07b-86a8-4ded-9de5-d9ee06d0ceb2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:10.132886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038c07b-86a8-4ded-9de5-d9ee06d0ceb2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:10.134259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038c07b-86a8-4ded-9de5-d9ee06d0ceb2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:10.139121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 393/794] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:10.140591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b42ae8-d7ca-4ce8-aac6-7f1b362f5711 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a038c07b-86a8-4ded-9de5-d9ee06d0ceb2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:10.156859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:10.157905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:49:10.158760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:10.161383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 402/795] 10.222.0.86 () {56 vars in 1129 bytes} [Mon Aug 29 18:49:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:10.162359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038c07b-86a8-4ded-9de5-d9ee06d0ceb2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:10.254867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b42ae8-d7ca-4ce8-aac6-7f1b362f5711 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:10.291824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49c195-e255-4689-8edb-55baa0b11ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:10.292826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c195-e255-4689-8edb-55baa0b11ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:10.293330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c195-e255-4689-8edb-55baa0b11ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:10.294431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c195-e255-4689-8edb-55baa0b11ec7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:10.301582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49c195-e255-4689-8edb-55baa0b11ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:10.303270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49c195-e255-4689-8edb-55baa0b11ec7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:10.308600 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038c07b-86a8-4ded-9de5-d9ee06d0ceb2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-cf49c195-e255-4689-8edb-55baa0b11ec7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:10.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:10.316943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:10.317833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:10.317833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:10.317833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:10.317833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:10.317833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:10.317833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:10.317833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:10.317833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:10.320386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 394/796] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:10.321365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49c195-e255-4689-8edb-55baa0b11ec7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:10.349550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3aa4df-b2a0-42fb-bf6a-0ca0011ec58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:10.350201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aa4df-b2a0-42fb-bf6a-0ca0011ec58b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:10.350503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aa4df-b2a0-42fb-bf6a-0ca0011ec58b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:10.350871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aa4df-b2a0-42fb-bf6a-0ca0011ec58b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:10.355169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:10.367105 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3aa4df-b2a0-42fb-bf6a-0ca0011ec58b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ef3aa4df-b2a0-42fb-bf6a-0ca0011ec58b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:10.374187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:10.375163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:10.376013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 403/797] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:49:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:10.376712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3aa4df-b2a0-42fb-bf6a-0ca0011ec58b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:10.436284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49c195-e255-4689-8edb-55baa0b11ec7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:10.473281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081b6cf-f55c-4ef9-86f3-683031f4043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:10.474718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081b6cf-f55c-4ef9-86f3-683031f4043e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:10.475285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081b6cf-f55c-4ef9-86f3-683031f4043e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:10.475407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:10.475827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081b6cf-f55c-4ef9-86f3-683031f4043e None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:10.476762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081b6cf-f55c-4ef9-86f3-683031f4043e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:10.477326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081b6cf-f55c-4ef9-86f3-683031f4043e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:10.479096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081b6cf-f55c-4ef9-86f3-683031f4043e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:10.488029 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 395/798] 10.222.0.86 () {56 vars in 1124 bytes} [Mon Aug 29 18:49:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:10.489291 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081b6cf-f55c-4ef9-86f3-683031f4043e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:10.515553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3aa4df-b2a0-42fb-bf6a-0ca0011ec58b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:10.551951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b3ef7-af9b-4368-8ff3-a7003cfe675e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:10.552509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b3ef7-af9b-4368-8ff3-a7003cfe675e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:10.552933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b3ef7-af9b-4368-8ff3-a7003cfe675e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:10.553038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b3ef7-af9b-4368-8ff3-a7003cfe675e None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:10.553737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b3ef7-af9b-4368-8ff3-a7003cfe675e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:10.554041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b3ef7-af9b-4368-8ff3-a7003cfe675e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:10.555805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b3ef7-af9b-4368-8ff3-a7003cfe675e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:10.580726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 404/799] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:10.581726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b3ef7-af9b-4368-8ff3-a7003cfe675e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:10.597785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:10.606723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081b6cf-f55c-4ef9-86f3-683031f4043e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:10.650916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c96cda-8c2c-407d-901c-2e7484c65852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:10.651921 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c96cda-8c2c-407d-901c-2e7484c65852 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:10.652404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c96cda-8c2c-407d-901c-2e7484c65852 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:10.652910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c96cda-8c2c-407d-901c-2e7484c65852 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:10.653789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c96cda-8c2c-407d-901c-2e7484c65852 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:10.654333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c96cda-8c2c-407d-901c-2e7484c65852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:10.656064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c96cda-8c2c-407d-901c-2e7484c65852 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:10.667290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 396/800] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:49:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:10.668094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c96cda-8c2c-407d-901c-2e7484c65852 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:10.734840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b3ef7-af9b-4368-8ff3-a7003cfe675e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:10.738675 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:10.786440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833c434-7b33-4622-b8f5-b04de0f24f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:10.787333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833c434-7b33-4622-b8f5-b04de0f24f83 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:10.787623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833c434-7b33-4622-b8f5-b04de0f24f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:10.787900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833c434-7b33-4622-b8f5-b04de0f24f83 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:10.788593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833c434-7b33-4622-b8f5-b04de0f24f83 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:10.789233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833c434-7b33-4622-b8f5-b04de0f24f83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:10.790794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833c434-7b33-4622-b8f5-b04de0f24f83 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:10.798665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 405/801] 10.222.0.86 () {56 vars in 1124 bytes} [Mon Aug 29 18:49:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:10.800103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833c434-7b33-4622-b8f5-b04de0f24f83 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:10.806355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c96cda-8c2c-407d-901c-2e7484c65852 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:10.852038 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64126f1-85df-4e79-b02d-6a59ec51639c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:10.852731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64126f1-85df-4e79-b02d-6a59ec51639c None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:10.853014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64126f1-85df-4e79-b02d-6a59ec51639c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:10.853295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64126f1-85df-4e79-b02d-6a59ec51639c None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/dd1b7173843b4da48e69da8b36d59226/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:10.855429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64126f1-85df-4e79-b02d-6a59ec51639c None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=dd1b7173843b4da48e69da8b36d59226, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:10.857414 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64126f1-85df-4e79-b02d-6a59ec51639c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:10.874393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 397/802] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:49:10 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/dd1b7173843b4da48e69da8b36d59226/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:49:10.875387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64126f1-85df-4e79-b02d-6a59ec51639c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:10.899817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:11.015812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64126f1-85df-4e79-b02d-6a59ec51639c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:11.045259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833c434-7b33-4622-b8f5-b04de0f24f83 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:11.060666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab7ba7-2c1e-46b7-93a5-15f9c14de522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:11.061678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab7ba7-2c1e-46b7-93a5-15f9c14de522 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:11.062225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab7ba7-2c1e-46b7-93a5-15f9c14de522 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:11.062731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab7ba7-2c1e-46b7-93a5-15f9c14de522 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:11.063655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab7ba7-2c1e-46b7-93a5-15f9c14de522 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:11.065287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab7ba7-2c1e-46b7-93a5-15f9c14de522 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:11.096441 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 398/803] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 18:49:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:11.120508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4601d-545e-4475-9d8d-7722ed6dcef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:11.121959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4601d-545e-4475-9d8d-7722ed6dcef3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:11.122445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4601d-545e-4475-9d8d-7722ed6dcef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:11.122937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4601d-545e-4475-9d8d-7722ed6dcef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:11.144207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4601d-545e-4475-9d8d-7722ed6dcef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:11.146150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4601d-545e-4475-9d8d-7722ed6dcef3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-26d4601d-545e-4475-9d8d-7722ed6dcef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:11.168038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:11.169630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:11.170547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:11.170547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:11.170547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:11.172486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 406/804] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:11.178525 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab7ba7-2c1e-46b7-93a5-15f9c14de522 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:11.277038 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:11.309243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab7ba7-2c1e-46b7-93a5-15f9c14de522 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:11.346007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce21a75-771a-4e60-971d-ddb52c1c0b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:11.346615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce21a75-771a-4e60-971d-ddb52c1c0b21 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:11.346903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce21a75-771a-4e60-971d-ddb52c1c0b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:11.347237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce21a75-771a-4e60-971d-ddb52c1c0b21 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:11.347893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce21a75-771a-4e60-971d-ddb52c1c0b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:11.348195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce21a75-771a-4e60-971d-ddb52c1c0b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:11.349557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce21a75-771a-4e60-971d-ddb52c1c0b21 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:11.362429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 399/805] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:11.367995 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4601d-545e-4475-9d8d-7722ed6dcef3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:11.374850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c53e13-db69-4f42-a785-4e6bbe894f4d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:11.375164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c53e13-db69-4f42-a785-4e6bbe894f4d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:11.375439 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c53e13-db69-4f42-a785-4e6bbe894f4d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:11.377055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 400/806] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:11.382086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9d127-cca3-4d22-953c-66b1b87bf566 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:11.382369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9d127-cca3-4d22-953c-66b1b87bf566 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:11.382647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9d127-cca3-4d22-953c-66b1b87bf566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:11.390510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:11.407049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-24e9d127-cca3-4d22-953c-66b1b87bf566 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:11.408262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-24e9d127-cca3-4d22-953c-66b1b87bf566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:11.498431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4601d-545e-4475-9d8d-7722ed6dcef3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:11.503391 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 401/807] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:11.537400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0adac-8a90-466e-9515-f23f633668b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:11.538318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0adac-8a90-466e-9515-f23f633668b3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:11.538776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0adac-8a90-466e-9515-f23f633668b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:11.539266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0adac-8a90-466e-9515-f23f633668b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:11.548398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0adac-8a90-466e-9515-f23f633668b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:11.550161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0adac-8a90-466e-9515-f23f633668b3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-58a0adac-8a90-466e-9515-f23f633668b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:11.562177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:11.563203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:11.564150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:11.564150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:11.564150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:11.564150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:11.564150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:11.564150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:11.564150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:11.564150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:11.564150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:11.564150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:11.566313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 407/808] 10.222.0.86 () {56 vars in 1117 bytes} [Mon Aug 29 18:49:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:11.575215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9d127-cca3-4d22-953c-66b1b87bf566 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:11.655129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:11.697020 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9d127-cca3-4d22-953c-66b1b87bf566 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:11.733900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a18c18-3d2c-41ca-83f7-eca54f52aba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:11.734522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a18c18-3d2c-41ca-83f7-eca54f52aba4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:11.734801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a18c18-3d2c-41ca-83f7-eca54f52aba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:11.735103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a18c18-3d2c-41ca-83f7-eca54f52aba4 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:11.735840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a18c18-3d2c-41ca-83f7-eca54f52aba4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:11.736145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a18c18-3d2c-41ca-83f7-eca54f52aba4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:11.737565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a18c18-3d2c-41ca-83f7-eca54f52aba4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:11.756242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 402/809] 10.222.0.86 () {56 vars in 1149 bytes} [Mon Aug 29 18:49:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:11.761731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0adac-8a90-466e-9515-f23f633668b3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:11.796314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b5dcb-b56b-4d27-8f2b-583d74e938e0 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:11.796498 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b5dcb-b56b-4d27-8f2b-583d74e938e0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:11.796783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b5dcb-b56b-4d27-8f2b-583d74e938e0 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:11.798572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 403/810] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:11.806369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aa65d-ec99-4465-9682-b73fd4b2fdb3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:11.806651 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aa65d-ec99-4465-9682-b73fd4b2fdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:11.807122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aa65d-ec99-4465-9682-b73fd4b2fdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:11.826973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-308aa65d-ec99-4465-9682-b73fd4b2fdb3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:11.828335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-308aa65d-ec99-4465-9682-b73fd4b2fdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:11.903250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 404/811] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:11.911522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:11.929691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308aa65d-ec99-4465-9682-b73fd4b2fdb3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:11.957628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0adac-8a90-466e-9515-f23f633668b3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:11.995064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:12.040671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ff563-8775-41b7-a73f-270174a15619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:12.041319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff563-8775-41b7-a73f-270174a15619 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:12.041597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff563-8775-41b7-a73f-270174a15619 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:12.041867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff563-8775-41b7-a73f-270174a15619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:12.051375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308aa65d-ec99-4465-9682-b73fd4b2fdb3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:12.061453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ff563-8775-41b7-a73f-270174a15619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:12.063067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ff563-8775-41b7-a73f-270174a15619 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-518ff563-8775-41b7-a73f-270174a15619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:12.085340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:12.086416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:12.087323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:12.087323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:12.087323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:12.087323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 408/812] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:12.092583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ff563-8775-41b7-a73f-270174a15619 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:12.096917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b3c8d-c1c7-4c8c-947e-9b12c7b91403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:12.097567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b3c8d-c1c7-4c8c-947e-9b12c7b91403 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:12.097845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b3c8d-c1c7-4c8c-947e-9b12c7b91403 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:12.098115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b3c8d-c1c7-4c8c-947e-9b12c7b91403 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:12.119342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b3c8d-c1c7-4c8c-947e-9b12c7b91403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2f8b3c8d-c1c7-4c8c-947e-9b12c7b91403 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:12.127250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:12.128171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:12.129012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 405/813] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:49:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:12.141172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b3c8d-c1c7-4c8c-947e-9b12c7b91403 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:12.223973 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ff563-8775-41b7-a73f-270174a15619 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:12.251507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:12.259576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc66e1d-af25-44bf-940f-63b1c4c15473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:12.260209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc66e1d-af25-44bf-940f-63b1c4c15473 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:12.260485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc66e1d-af25-44bf-940f-63b1c4c15473 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:12.260753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc66e1d-af25-44bf-940f-63b1c4c15473 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:12.261428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc66e1d-af25-44bf-940f-63b1c4c15473 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:12.261728 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc66e1d-af25-44bf-940f-63b1c4c15473 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:12.263184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc66e1d-af25-44bf-940f-63b1c4c15473 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:12.275664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 409/814] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:12.276579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc66e1d-af25-44bf-940f-63b1c4c15473 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:12.279888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b3c8d-c1c7-4c8c-947e-9b12c7b91403 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:12.294865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:12.322945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b309d2-1bde-4d9d-ba47-e29c0417969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:12.324181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b309d2-1bde-4d9d-ba47-e29c0417969d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:12.325177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b309d2-1bde-4d9d-ba47-e29c0417969d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:12.325934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b309d2-1bde-4d9d-ba47-e29c0417969d None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:12.326869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b309d2-1bde-4d9d-ba47-e29c0417969d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:12.327560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b309d2-1bde-4d9d-ba47-e29c0417969d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:12.329538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b309d2-1bde-4d9d-ba47-e29c0417969d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:12.346437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 406/815] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:49:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:12.348270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b309d2-1bde-4d9d-ba47-e29c0417969d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:12.384849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc66e1d-af25-44bf-940f-63b1c4c15473 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:12.421636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547e76a-ebc0-4edd-a59e-7026b144afbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:12.422401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547e76a-ebc0-4edd-a59e-7026b144afbf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:12.422774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547e76a-ebc0-4edd-a59e-7026b144afbf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:12.423165 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547e76a-ebc0-4edd-a59e-7026b144afbf None admin] PATH_INFO: `/v3/services/network` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:12.423836 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547e76a-ebc0-4edd-a59e-7026b144afbf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:12.425292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547e76a-ebc0-4edd-a59e-7026b144afbf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1547e76a-ebc0-4edd-a59e-7026b144afbf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:12.434357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:12.435583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:49:12.436720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:12.443099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 410/816] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:12 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:12.444330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547e76a-ebc0-4edd-a59e-7026b144afbf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:12.508467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:12.571554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547e76a-ebc0-4edd-a59e-7026b144afbf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:12.608494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b309d2-1bde-4d9d-ba47-e29c0417969d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:12.612688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a283ef-62ca-4c60-9e4f-9b65781cb686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:12.613427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a283ef-62ca-4c60-9e4f-9b65781cb686 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:12.613802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a283ef-62ca-4c60-9e4f-9b65781cb686 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:12.614168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a283ef-62ca-4c60-9e4f-9b65781cb686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:12.629115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a283ef-62ca-4c60-9e4f-9b65781cb686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:12.631252 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a283ef-62ca-4c60-9e4f-9b65781cb686 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:12.645290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01b3d2-c3b4-4757-95e4-995d15299d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:12.645914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01b3d2-c3b4-4757-95e4-995d15299d4e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:12.646192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01b3d2-c3b4-4757-95e4-995d15299d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:12.646468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01b3d2-c3b4-4757-95e4-995d15299d4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-76a283ef-62ca-4c60-9e4f-9b65781cb686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:12.650194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:12.651236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:12.652134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:12.652134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:12.652134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:12.652134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 411/817] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:12.653518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a283ef-62ca-4c60-9e4f-9b65781cb686 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:12.653695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01b3d2-c3b4-4757-95e4-995d15299d4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:12.655276 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01b3d2-c3b4-4757-95e4-995d15299d4e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-db01b3d2-c3b4-4757-95e4-995d15299d4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:12.666865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:12.669016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:12.671126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:12.671126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:12.671126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:12.671126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:12.671126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:12.671126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:12.671126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:12.671126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:12.671126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 407/818] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:12.671126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01b3d2-c3b4-4757-95e4-995d15299d4e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:12.711426 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:12.780259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a283ef-62ca-4c60-9e4f-9b65781cb686 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:12.783746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:12.795439 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01b3d2-c3b4-4757-95e4-995d15299d4e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:12.818007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace73073-f932-406e-9bae-2586f0c41f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:12.819139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace73073-f932-406e-9bae-2586f0c41f5b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:12.819566 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace73073-f932-406e-9bae-2586f0c41f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:12.820089 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace73073-f932-406e-9bae-2586f0c41f5b None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:12.820862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace73073-f932-406e-9bae-2586f0c41f5b None admin] RBAC: Adding query filter params (name=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:12.821344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace73073-f932-406e-9bae-2586f0c41f5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:12.822844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace73073-f932-406e-9bae-2586f0c41f5b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:12.828553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 412/819] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:12.829583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace73073-f932-406e-9bae-2586f0c41f5b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:12.838901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9946c-782c-48b0-a80a-772bbce671cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:12.839738 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9946c-782c-48b0-a80a-772bbce671cc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:12.840165 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9946c-782c-48b0-a80a-772bbce671cc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:12.841101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9946c-782c-48b0-a80a-772bbce671cc None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:12.842401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f9946c-782c-48b0-a80a-772bbce671cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:12.842866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f9946c-782c-48b0-a80a-772bbce671cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:12.844934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f9946c-782c-48b0-a80a-772bbce671cc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:12.861725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 408/820] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:12.863149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9946c-782c-48b0-a80a-772bbce671cc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:12.955382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace73073-f932-406e-9bae-2586f0c41f5b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:12.990938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:12.991923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fd872-d655-4fe7-a3c9-f8cb55555452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:12.992541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fd872-d655-4fe7-a3c9-f8cb55555452 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:12.993096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fd872-d655-4fe7-a3c9-f8cb55555452 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:12.993096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fd872-d655-4fe7-a3c9-f8cb55555452 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:12.993805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fd872-d655-4fe7-a3c9-f8cb55555452 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:12.994259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fd872-d655-4fe7-a3c9-f8cb55555452 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:12.995639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fd872-d655-4fe7-a3c9-f8cb55555452 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:13.001714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9946c-782c-48b0-a80a-772bbce671cc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:13.009781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 413/821] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:13.010717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fd872-d655-4fe7-a3c9-f8cb55555452 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:13.040646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:13.053480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c10de6-c95f-437b-ad1b-96fed911cf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:13.054394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c10de6-c95f-437b-ad1b-96fed911cf5c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:13.054919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c10de6-c95f-437b-ad1b-96fed911cf5c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:13.055503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c10de6-c95f-437b-ad1b-96fed911cf5c None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:13.056328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c10de6-c95f-437b-ad1b-96fed911cf5c None admin] RBAC: Adding query filter params (type=network) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:13.057363 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c10de6-c95f-437b-ad1b-96fed911cf5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:13.060031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c10de6-c95f-437b-ad1b-96fed911cf5c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:13.072703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 409/822] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:13.073620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c10de6-c95f-437b-ad1b-96fed911cf5c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:13.133939 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fd872-d655-4fe7-a3c9-f8cb55555452 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:13.170590 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75f520-636f-4265-8a06-3726b6a93d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:13.171313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75f520-636f-4265-8a06-3726b6a93d03 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:13.171590 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75f520-636f-4265-8a06-3726b6a93d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:13.171859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75f520-636f-4265-8a06-3726b6a93d03 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:13.181149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75f520-636f-4265-8a06-3726b6a93d03 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:13.183674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75f520-636f-4265-8a06-3726b6a93d03 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:13.201840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c10de6-c95f-437b-ad1b-96fed911cf5c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9c75f520-636f-4265-8a06-3726b6a93d03 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:13.202143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:13.203103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:13.204067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:13.204067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:13.204067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:13.204067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:13.204067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:13.204067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:13.204067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:13.204067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:13.204067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:13.204067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:13.207563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 414/823] 10.222.0.86 () {56 vars in 1093 bytes} [Mon Aug 29 18:49:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:13.208730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75f520-636f-4265-8a06-3726b6a93d03 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:13.240386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f8eba-4751-4aa8-acea-3082c48fd228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:13.241050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f8eba-4751-4aa8-acea-3082c48fd228 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:13.241326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f8eba-4751-4aa8-acea-3082c48fd228 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:13.241592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f8eba-4751-4aa8-acea-3082c48fd228 None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/7634f750a07143a8b8d91cf0c8e78e16/roles/000d8f9e15c142349b2f85e8ca441e96` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:13.257994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f8eba-4751-4aa8-acea-3082c48fd228 None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=7634f750a07143a8b8d91cf0c8e78e16, role_id=000d8f9e15c142349b2f85e8ca441e96)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:13.258802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f8eba-4751-4aa8-acea-3082c48fd228 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:13.273588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 410/824] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:49:13 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/7634f750a07143a8b8d91cf0c8e78e16/roles/000d8f9e15c142349b2f85e8ca441e96 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:49:13.274818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f8eba-4751-4aa8-acea-3082c48fd228 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:13.288416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:13.323099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:13.337404 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75f520-636f-4265-8a06-3726b6a93d03 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:13.380846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6dd28-6c3d-4027-a9e1-cea5b91a379f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:13.381473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6dd28-6c3d-4027-a9e1-cea5b91a379f None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:13.381744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6dd28-6c3d-4027-a9e1-cea5b91a379f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:13.382008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6dd28-6c3d-4027-a9e1-cea5b91a379f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:13.382684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6dd28-6c3d-4027-a9e1-cea5b91a379f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:13.384562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6dd28-6c3d-4027-a9e1-cea5b91a379f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:13.404509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f8eba-4751-4aa8-acea-3082c48fd228 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:13.423218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 415/825] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 18:49:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:13.492719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32aac1-83f2-4dc2-9a7c-767d75cd8357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:13.493799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32aac1-83f2-4dc2-9a7c-767d75cd8357 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:13.494281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32aac1-83f2-4dc2-9a7c-767d75cd8357 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:13.494778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32aac1-83f2-4dc2-9a7c-767d75cd8357 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:13.495759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32aac1-83f2-4dc2-9a7c-767d75cd8357 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:13.496288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32aac1-83f2-4dc2-9a7c-767d75cd8357 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:13.498032 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32aac1-83f2-4dc2-9a7c-767d75cd8357 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:13.504394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153367-371b-49b4-954b-5626737eeb62 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:13.504731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153367-371b-49b4-954b-5626737eeb62 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:13.504959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153367-371b-49b4-954b-5626737eeb62 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:13.506592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 416/826] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:13.512251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb68273-8c17-4d3b-8735-57516c2de3f8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:13.512553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb68273-8c17-4d3b-8735-57516c2de3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:13.512801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb68273-8c17-4d3b-8735-57516c2de3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:13.530271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 411/827] 10.222.0.86 () {56 vars in 1133 bytes} [Mon Aug 29 18:49:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:13.533048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8eb68273-8c17-4d3b-8735-57516c2de3f8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:13.534287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8eb68273-8c17-4d3b-8735-57516c2de3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:13.535815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32aac1-83f2-4dc2-9a7c-767d75cd8357 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:13.622044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:13.627452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 417/828] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:13.628478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0adc73-4544-49ab-86cc-00f31ff91b5a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:13.628763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0adc73-4544-49ab-86cc-00f31ff91b5a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:13.629034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0adc73-4544-49ab-86cc-00f31ff91b5a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:13.630470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 418/829] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:13.636691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fbba1-a093-43dc-a4f4-4cc898679fd9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:13.636913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fbba1-a093-43dc-a4f4-4cc898679fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:13.637719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fbba1-a093-43dc-a4f4-4cc898679fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:13.657250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5a4fbba1-a093-43dc-a4f4-4cc898679fd9 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:13.658441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5a4fbba1-a093-43dc-a4f4-4cc898679fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:13.679330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32aac1-83f2-4dc2-9a7c-767d75cd8357 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:13.725556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da7e0e-73b4-4a80-a932-6eaad698584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:13.726511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da7e0e-73b4-4a80-a932-6eaad698584a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:13.727030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da7e0e-73b4-4a80-a932-6eaad698584a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:13.727586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da7e0e-73b4-4a80-a932-6eaad698584a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:13.746749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 419/830] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:13.807047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da7e0e-73b4-4a80-a932-6eaad698584a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:13.808493 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da7e0e-73b4-4a80-a932-6eaad698584a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:13.871628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:13.915208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 412/831] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 18:49:13.920760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fbba1-a093-43dc-a4f4-4cc898679fd9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:13.925692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:13.931970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da7e0e-73b4-4a80-a932-6eaad698584a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:14.070519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fbba1-a093-43dc-a4f4-4cc898679fd9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.075583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da7e0e-73b4-4a80-a932-6eaad698584a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.110301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08999c58-6348-4cda-8c75-0437f8838375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:14.110633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11caae50-f32f-4550-afcb-e9365e11c90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:14.110934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08999c58-6348-4cda-8c75-0437f8838375 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:14.111242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08999c58-6348-4cda-8c75-0437f8838375 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:14.111515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08999c58-6348-4cda-8c75-0437f8838375 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:14.112136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caae50-f32f-4550-afcb-e9365e11c90e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:14.112602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caae50-f32f-4550-afcb-e9365e11c90e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:14.113058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caae50-f32f-4550-afcb-e9365e11c90e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:14.122299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08999c58-6348-4cda-8c75-0437f8838375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:14.124511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11caae50-f32f-4550-afcb-e9365e11c90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:14.126153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11caae50-f32f-4550-afcb-e9365e11c90e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-08999c58-6348-4cda-8c75-0437f8838375 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:14.127305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:14.128222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:14.129078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 413/832] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:49:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:14.129853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08999c58-6348-4cda-8c75-0437f8838375 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-11caae50-f32f-4550-afcb-e9365e11c90e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:14.148663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:14.149646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:14.150538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:14.150538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:14.150538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:14.152547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 420/833] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:14.153462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11caae50-f32f-4550-afcb-e9365e11c90e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:14.160655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:14.217241 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:14.242322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08999c58-6348-4cda-8c75-0437f8838375 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.263930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11caae50-f32f-4550-afcb-e9365e11c90e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.278006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdd8f2-3e48-4952-8e82-5d1e7e175d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:14.278638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdd8f2-3e48-4952-8e82-5d1e7e175d5f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:14.278919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdd8f2-3e48-4952-8e82-5d1e7e175d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:14.279219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdd8f2-3e48-4952-8e82-5d1e7e175d5f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:14.279791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdd8f2-3e48-4952-8e82-5d1e7e175d5f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:14.281425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdd8f2-3e48-4952-8e82-5d1e7e175d5f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-92bdd8f2-3e48-4952-8e82-5d1e7e175d5f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:14.292750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:14.293792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:49:14.294641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:14.295414 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 414/834] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:49:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:14.295982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdd8f2-3e48-4952-8e82-5d1e7e175d5f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:14.303316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f602af-2157-4563-9006-5143cfd7bfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:14.303935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f602af-2157-4563-9006-5143cfd7bfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:14.304215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f602af-2157-4563-9006-5143cfd7bfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:14.304488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f602af-2157-4563-9006-5143cfd7bfa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:14.305153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f602af-2157-4563-9006-5143cfd7bfa8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:14.305456 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f602af-2157-4563-9006-5143cfd7bfa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:14.307297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f602af-2157-4563-9006-5143cfd7bfa8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:14.318302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 421/835] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:49:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:14.319275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f602af-2157-4563-9006-5143cfd7bfa8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:14.393394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:14.408723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdd8f2-3e48-4952-8e82-5d1e7e175d5f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.420770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f602af-2157-4563-9006-5143cfd7bfa8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.441131 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:14.445353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb8d36-081f-4968-82e2-596c750581c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:14.445959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb8d36-081f-4968-82e2-596c750581c4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:14.446237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb8d36-081f-4968-82e2-596c750581c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:14.446502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb8d36-081f-4968-82e2-596c750581c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:14.447206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb8d36-081f-4968-82e2-596c750581c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:14.447488 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb8d36-081f-4968-82e2-596c750581c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:14.448843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb8d36-081f-4968-82e2-596c750581c4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:14.460678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 415/836] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:14.461361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b6a82-3412-460c-bc00-a9a46b78fc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:14.461496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb8d36-081f-4968-82e2-596c750581c4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:14.462216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b6a82-3412-460c-bc00-a9a46b78fc4a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:14.462626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b6a82-3412-460c-bc00-a9a46b78fc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:14.463054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b6a82-3412-460c-bc00-a9a46b78fc4a None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:14.463828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b6a82-3412-460c-bc00-a9a46b78fc4a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:14.464259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b6a82-3412-460c-bc00-a9a46b78fc4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:14.465810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b6a82-3412-460c-bc00-a9a46b78fc4a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:14.471663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 422/837] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:49:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:14.476299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b6a82-3412-460c-bc00-a9a46b78fc4a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:14.573322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb8d36-081f-4968-82e2-596c750581c4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.582145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b6a82-3412-460c-bc00-a9a46b78fc4a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.614042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a607e-1760-4bb9-9bb3-a6199d1c29a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:14.614675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a607e-1760-4bb9-9bb3-a6199d1c29a1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:14.614948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a607e-1760-4bb9-9bb3-a6199d1c29a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:14.615268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a607e-1760-4bb9-9bb3-a6199d1c29a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:14.619034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03d550-2a27-4173-a21a-3e39b999a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:14.619807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03d550-2a27-4173-a21a-3e39b999a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:14.620224 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03d550-2a27-4173-a21a-3e39b999a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:14.620633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03d550-2a27-4173-a21a-3e39b999a5e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:14.626232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a607e-1760-4bb9-9bb3-a6199d1c29a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:14.627694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a607e-1760-4bb9-9bb3-a6199d1c29a1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:14.628750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03d550-2a27-4173-a21a-3e39b999a5e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:14.630312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03d550-2a27-4173-a21a-3e39b999a5e9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:14.634650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7e03d550-2a27-4173-a21a-3e39b999a5e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:14.641246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:14.642134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:14.648003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:14.648003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:14.648003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:14.648003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:14.648003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:14.648003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:14.648003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:14.648003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:14.648003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 423/838] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:14.648003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03d550-2a27-4173-a21a-3e39b999a5e9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-bf8a607e-1760-4bb9-9bb3-a6199d1c29a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:14.648569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:14.649420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:14.650285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:14.650285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:14.650285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:14.650285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 416/839] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:14.650518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a607e-1760-4bb9-9bb3-a6199d1c29a1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:14.708215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:14.757008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a607e-1760-4bb9-9bb3-a6199d1c29a1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.767399 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03d550-2a27-4173-a21a-3e39b999a5e9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.796651 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabbb2a-3274-4ec2-8106-3088716bab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:14.797266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabbb2a-3274-4ec2-8106-3088716bab19 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:14.797542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabbb2a-3274-4ec2-8106-3088716bab19 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:14.798004 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabbb2a-3274-4ec2-8106-3088716bab19 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:14.798486 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabbb2a-3274-4ec2-8106-3088716bab19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:14.798866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabbb2a-3274-4ec2-8106-3088716bab19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:14.800218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabbb2a-3274-4ec2-8106-3088716bab19 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:14.802903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17980d10-35da-43ad-b8e1-29d5cc603a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:14.803528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17980d10-35da-43ad-b8e1-29d5cc603a05 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:14.803804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17980d10-35da-43ad-b8e1-29d5cc603a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:14.804072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17980d10-35da-43ad-b8e1-29d5cc603a05 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:14.804689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17980d10-35da-43ad-b8e1-29d5cc603a05 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:14.804987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17980d10-35da-43ad-b8e1-29d5cc603a05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:14.806317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17980d10-35da-43ad-b8e1-29d5cc603a05 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:14.819548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 424/840] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:49:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:14.819548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17980d10-35da-43ad-b8e1-29d5cc603a05 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:14.819857 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 417/841] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:14.819857 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabbb2a-3274-4ec2-8106-3088716bab19 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:14.895265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:14.945880 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabbb2a-3274-4ec2-8106-3088716bab19 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.972166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17980d10-35da-43ad-b8e1-29d5cc603a05 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:14.986329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:14.999046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd6fed-3d0e-42fc-8e75-d2b981cab6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:14.999046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd6fed-3d0e-42fc-8e75-d2b981cab6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:14.999487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd6fed-3d0e-42fc-8e75-d2b981cab6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:15.000606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd6fed-3d0e-42fc-8e75-d2b981cab6b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:15.000606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd6fed-3d0e-42fc-8e75-d2b981cab6b5 None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=045b22d046c549f2b53d2ed73b94f936) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:15.001013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd6fed-3d0e-42fc-8e75-d2b981cab6b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:15.003092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd6fed-3d0e-42fc-8e75-d2b981cab6b5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:15.011107 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 418/842] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:14 2022] GET /identity/v3/role_assignments?user.id=045b22d046c549f2b53d2ed73b94f936&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:15.021147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50825c54-0f9f-4da6-a39c-b8e53f03513b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:15.022072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50825c54-0f9f-4da6-a39c-b8e53f03513b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:15.022506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50825c54-0f9f-4da6-a39c-b8e53f03513b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:15.022931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50825c54-0f9f-4da6-a39c-b8e53f03513b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:15.023898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50825c54-0f9f-4da6-a39c-b8e53f03513b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:15.024386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50825c54-0f9f-4da6-a39c-b8e53f03513b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:15.025961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50825c54-0f9f-4da6-a39c-b8e53f03513b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:15.041433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 425/843] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:15.048694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd6fed-3d0e-42fc-8e75-d2b981cab6b5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:15.170693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd6fed-3d0e-42fc-8e75-d2b981cab6b5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:15.195159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:15.208678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430b586-2ed3-41d1-a461-9d45da96d715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:15.209321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430b586-2ed3-41d1-a461-9d45da96d715 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:15.209593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430b586-2ed3-41d1-a461-9d45da96d715 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:15.209863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430b586-2ed3-41d1-a461-9d45da96d715 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:15.218485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430b586-2ed3-41d1-a461-9d45da96d715 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:15.219954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430b586-2ed3-41d1-a461-9d45da96d715 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0430b586-2ed3-41d1-a461-9d45da96d715 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:15.231640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:15.232573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:15.233576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:15.233576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:15.233576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:15.233576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:15.233576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:15.233576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:15.233576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:15.233576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:15.233576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:49:15.233576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:15.233576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 419/844] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:49:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:15.238765 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50825c54-0f9f-4da6-a39c-b8e53f03513b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:15.353327 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50825c54-0f9f-4da6-a39c-b8e53f03513b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:15.390680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ee4cd-1d59-44d1-8d20-1f62bbb705d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:15.391331 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ee4cd-1d59-44d1-8d20-1f62bbb705d5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:15.391610 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ee4cd-1d59-44d1-8d20-1f62bbb705d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:15.391882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ee4cd-1d59-44d1-8d20-1f62bbb705d5 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:15.392615 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ee4cd-1d59-44d1-8d20-1f62bbb705d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:15.392923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ee4cd-1d59-44d1-8d20-1f62bbb705d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:15.394342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ee4cd-1d59-44d1-8d20-1f62bbb705d5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:15.416441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 426/845] 10.222.0.86 () {56 vars in 1135 bytes} [Mon Aug 29 18:49:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:15.422397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430b586-2ed3-41d1-a461-9d45da96d715 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:15.433219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2e853-4949-4f68-aef5-858fa5f37ac6 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:15.433694 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2e853-4949-4f68-aef5-858fa5f37ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:15.433771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2e853-4949-4f68-aef5-858fa5f37ac6 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:15.436417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 427/846] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:15.440340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee0b80-436c-400c-b24d-aba9901f4d16 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:15.440623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee0b80-436c-400c-b24d-aba9901f4d16 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:15.440893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee0b80-436c-400c-b24d-aba9901f4d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:15.459142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fcee0b80-436c-400c-b24d-aba9901f4d16 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:15.460295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fcee0b80-436c-400c-b24d-aba9901f4d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:15.478338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:15.543146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430b586-2ed3-41d1-a461-9d45da96d715 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:15.547064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 428/847] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:15.585758 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db18123-59ad-40d8-a9af-db8868e98725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:15.586369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db18123-59ad-40d8-a9af-db8868e98725 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:15.586643 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db18123-59ad-40d8-a9af-db8868e98725 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:15.586908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db18123-59ad-40d8-a9af-db8868e98725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:15.636815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:15.658117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db18123-59ad-40d8-a9af-db8868e98725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:15.660416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db18123-59ad-40d8-a9af-db8868e98725 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:15.786147 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 420/848] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:49:15.791910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee0b80-436c-400c-b24d-aba9901f4d16 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:15.857696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db18123-59ad-40d8-a9af-db8868e98725 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:15.898500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee0b80-436c-400c-b24d-aba9901f4d16 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:15.936091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00a171-70a6-432f-8c7c-9e6e3f306400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:15.937133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00a171-70a6-432f-8c7c-9e6e3f306400 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:15.937653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00a171-70a6-432f-8c7c-9e6e3f306400 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:15.938175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00a171-70a6-432f-8c7c-9e6e3f306400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:15.941491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:15.950189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00a171-70a6-432f-8c7c-9e6e3f306400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:15.951928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00a171-70a6-432f-8c7c-9e6e3f306400 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:15.960359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db18123-59ad-40d8-a9af-db8868e98725 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2c00a171-70a6-432f-8c7c-9e6e3f306400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:15.968806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:15.969776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:15.970686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:15.970686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:15.970686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:15.972959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 429/849] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:15.978852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00a171-70a6-432f-8c7c-9e6e3f306400 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:15.998816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a67747-ed1d-44db-a939-8213c40ccf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:15.999501 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a67747-ed1d-44db-a939-8213c40ccf48 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:15.999780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a67747-ed1d-44db-a939-8213c40ccf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:16.000054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a67747-ed1d-44db-a939-8213c40ccf48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:16.007107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:16.010566 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a67747-ed1d-44db-a939-8213c40ccf48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-43a67747-ed1d-44db-a939-8213c40ccf48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:16.015382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:16.016314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:49:16.017178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:16.021281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 421/850] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:49:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:16.022476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a67747-ed1d-44db-a939-8213c40ccf48 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:16.130047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00a171-70a6-432f-8c7c-9e6e3f306400 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:16.139540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a67747-ed1d-44db-a939-8213c40ccf48 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:16.175872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16548e0a-15c8-4dbd-9342-0bddaea1d6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:16.176498 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a089a-a9b2-4c9c-8305-e04d1d089b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:16.176969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16548e0a-15c8-4dbd-9342-0bddaea1d6fc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:16.177270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a089a-a9b2-4c9c-8305-e04d1d089b99 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:16.177616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a089a-a9b2-4c9c-8305-e04d1d089b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:16.177817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16548e0a-15c8-4dbd-9342-0bddaea1d6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:16.177971 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a089a-a9b2-4c9c-8305-e04d1d089b99 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:16.178406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16548e0a-15c8-4dbd-9342-0bddaea1d6fc None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:16.178830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a089a-a9b2-4c9c-8305-e04d1d089b99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:16.179231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a089a-a9b2-4c9c-8305-e04d1d089b99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:16.179711 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16548e0a-15c8-4dbd-9342-0bddaea1d6fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:16.180186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16548e0a-15c8-4dbd-9342-0bddaea1d6fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:16.180855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a089a-a9b2-4c9c-8305-e04d1d089b99 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:16.181904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16548e0a-15c8-4dbd-9342-0bddaea1d6fc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:16.194724 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 422/851] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:49:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:16.195985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a089a-a9b2-4c9c-8305-e04d1d089b99 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:16.197471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 430/852] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:16.200913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:16.202797 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16548e0a-15c8-4dbd-9342-0bddaea1d6fc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:16.283585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:16.309210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16548e0a-15c8-4dbd-9342-0bddaea1d6fc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:16.333449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a089a-a9b2-4c9c-8305-e04d1d089b99 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:16.345804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349b967-d622-4bb5-864b-ed93e5373d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:16.346552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349b967-d622-4bb5-864b-ed93e5373d59 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:16.346923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349b967-d622-4bb5-864b-ed93e5373d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:16.347310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349b967-d622-4bb5-864b-ed93e5373d59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:16.354545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349b967-d622-4bb5-864b-ed93e5373d59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:16.356772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349b967-d622-4bb5-864b-ed93e5373d59 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1349b967-d622-4bb5-864b-ed93e5373d59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:16.369737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:16.370832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:16.372023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:16.372023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:16.372023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:16.372023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:16.372023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:16.372023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:16.372023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:16.372023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:16.377843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 431/853] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:16.379110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349b967-d622-4bb5-864b-ed93e5373d59 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:16.381668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527ace7-2271-49fe-84b1-78bcb615d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:16.382669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527ace7-2271-49fe-84b1-78bcb615d72e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:16.383690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527ace7-2271-49fe-84b1-78bcb615d72e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:16.383690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527ace7-2271-49fe-84b1-78bcb615d72e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:16.400483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527ace7-2271-49fe-84b1-78bcb615d72e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:16.403213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527ace7-2271-49fe-84b1-78bcb615d72e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2527ace7-2271-49fe-84b1-78bcb615d72e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:16.419367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:16.420353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:16.421250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:16.421250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:16.421250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:16.425847 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 423/854] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:16.427166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527ace7-2271-49fe-84b1-78bcb615d72e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:16.492337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349b967-d622-4bb5-864b-ed93e5373d59 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:16.499312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:16.531408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:16.539060 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d97c76-aef3-4c7a-b9a9-e3c29b7b0f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:16.539884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d97c76-aef3-4c7a-b9a9-e3c29b7b0f89 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:16.540295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d97c76-aef3-4c7a-b9a9-e3c29b7b0f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:16.540705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d97c76-aef3-4c7a-b9a9-e3c29b7b0f89 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:16.541616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d97c76-aef3-4c7a-b9a9-e3c29b7b0f89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:16.542056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d97c76-aef3-4c7a-b9a9-e3c29b7b0f89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:16.544272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d97c76-aef3-4c7a-b9a9-e3c29b7b0f89 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:16.550927 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527ace7-2271-49fe-84b1-78bcb615d72e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:16.558313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 432/855] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:16.564390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d97c76-aef3-4c7a-b9a9-e3c29b7b0f89 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:16.602317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c7585-4e89-4a91-a8c9-94d288a3f852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:16.604258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c7585-4e89-4a91-a8c9-94d288a3f852 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:16.604754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c7585-4e89-4a91-a8c9-94d288a3f852 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:16.605234 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c7585-4e89-4a91-a8c9-94d288a3f852 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:16.606248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c7585-4e89-4a91-a8c9-94d288a3f852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:16.606754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c7585-4e89-4a91-a8c9-94d288a3f852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:16.608914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c7585-4e89-4a91-a8c9-94d288a3f852 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:16.627740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 424/856] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:16.634802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c7585-4e89-4a91-a8c9-94d288a3f852 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:16.686575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d97c76-aef3-4c7a-b9a9-e3c29b7b0f89 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:16.736615 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a2df6-96e7-40a9-862e-5cfe3bf54972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:16.737523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a2df6-96e7-40a9-862e-5cfe3bf54972 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:16.737989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a2df6-96e7-40a9-862e-5cfe3bf54972 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:16.738338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a2df6-96e7-40a9-862e-5cfe3bf54972 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:16.739413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a2df6-96e7-40a9-862e-5cfe3bf54972 None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=dd1b7173843b4da48e69da8b36d59226) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:16.740138 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a2df6-96e7-40a9-862e-5cfe3bf54972 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:16.742161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a2df6-96e7-40a9-862e-5cfe3bf54972 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:16.748928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c7585-4e89-4a91-a8c9-94d288a3f852 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:16.754317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 433/857] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:16 2022] GET /identity/v3/role_assignments?user.id=dd1b7173843b4da48e69da8b36d59226&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:16.773963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:16.795721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e515a-063d-4455-95e6-9c486bf054d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:16.796702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e515a-063d-4455-95e6-9c486bf054d5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:16.797528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e515a-063d-4455-95e6-9c486bf054d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:16.797528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e515a-063d-4455-95e6-9c486bf054d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:16.806526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e515a-063d-4455-95e6-9c486bf054d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:16.808000 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e515a-063d-4455-95e6-9c486bf054d5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-bb1e515a-063d-4455-95e6-9c486bf054d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:16.821664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:16.822682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:16.823663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:16.823663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:16.823663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:16.823663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:16.823663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:16.823663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:16.823663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:16.823663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:16.823663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:16.823663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:16.827177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 425/858] 10.222.0.86 () {56 vars in 1117 bytes} [Mon Aug 29 18:49:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:16.829917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a2df6-96e7-40a9-862e-5cfe3bf54972 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:16.870719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa96119-08c6-44cb-b74a-123dffd6b057 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:16.871072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa96119-08c6-44cb-b74a-123dffd6b057 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:16.871335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa96119-08c6-44cb-b74a-123dffd6b057 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:16.873007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 426/859] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:16.878726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96837617-de7e-4530-bf75-6b5e3d6bd99c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:16.879079 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96837617-de7e-4530-bf75-6b5e3d6bd99c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:16.879324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96837617-de7e-4530-bf75-6b5e3d6bd99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:16.897839 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:16.899929 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-96837617-de7e-4530-bf75-6b5e3d6bd99c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:16.901209 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-96837617-de7e-4530-bf75-6b5e3d6bd99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:16.968936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a2df6-96e7-40a9-862e-5cfe3bf54972 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:16.995205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 427/860] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:17.015920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c545e-3b22-4a22-b5cf-b7e30a3f58f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:17.016912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c545e-3b22-4a22-b5cf-b7e30a3f58f6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:17.017447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c545e-3b22-4a22-b5cf-b7e30a3f58f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:17.017920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c545e-3b22-4a22-b5cf-b7e30a3f58f6 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:17.018916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c545e-3b22-4a22-b5cf-b7e30a3f58f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:17.019448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c545e-3b22-4a22-b5cf-b7e30a3f58f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:17.021161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c545e-3b22-4a22-b5cf-b7e30a3f58f6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:17.040091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 434/861] 10.222.0.86 () {56 vars in 1149 bytes} [Mon Aug 29 18:49:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:17.045629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96837617-de7e-4530-bf75-6b5e3d6bd99c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:17.070657 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27438a-693e-415f-93f2-78c07804c293 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:17.071288 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27438a-693e-415f-93f2-78c07804c293 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:17.071756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27438a-693e-415f-93f2-78c07804c293 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:17.073672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 435/862] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:17.078973 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aabfed-fc97-48b9-bf7a-7b157d3fb74a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:17.079546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aabfed-fc97-48b9-bf7a-7b157d3fb74a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:17.080076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aabfed-fc97-48b9-bf7a-7b157d3fb74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:17.100829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-28aabfed-fc97-48b9-bf7a-7b157d3fb74a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:17.102513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-28aabfed-fc97-48b9-bf7a-7b157d3fb74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:17.157633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96837617-de7e-4530-bf75-6b5e3d6bd99c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:17.165859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:17.187264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 436/863] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:17.196710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc086379-7102-4775-b047-63ea5d6b4829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:17.197595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc086379-7102-4775-b047-63ea5d6b4829 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:17.198114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc086379-7102-4775-b047-63ea5d6b4829 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:17.198599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc086379-7102-4775-b047-63ea5d6b4829 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:17.271482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc086379-7102-4775-b047-63ea5d6b4829 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:17.272946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aabfed-fc97-48b9-bf7a-7b157d3fb74a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:17.273371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc086379-7102-4775-b047-63ea5d6b4829 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:17.332472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:17.401629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 428/864] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:49:17.407451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aabfed-fc97-48b9-bf7a-7b157d3fb74a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:17.408514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc086379-7102-4775-b047-63ea5d6b4829 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:17.427341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:17.454469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf5b7d-8705-4591-8137-4b9a646e51e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:17.455285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf5b7d-8705-4591-8137-4b9a646e51e7 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:17.455660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf5b7d-8705-4591-8137-4b9a646e51e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:17.456027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf5b7d-8705-4591-8137-4b9a646e51e7 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:17.456936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf5b7d-8705-4591-8137-4b9a646e51e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:17.458303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf5b7d-8705-4591-8137-4b9a646e51e7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:17.479740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 437/865] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:49:17 2022] POST /identity/v3/services => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:49:17.481472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf5b7d-8705-4591-8137-4b9a646e51e7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:17.531863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc086379-7102-4775-b047-63ea5d6b4829 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:17.592611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33b868-23ce-4f7a-96a4-acda07ead6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:17.593319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33b868-23ce-4f7a-96a4-acda07ead6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:17.593599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33b868-23ce-4f7a-96a4-acda07ead6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:17.593867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33b868-23ce-4f7a-96a4-acda07ead6d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:17.608824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33b868-23ce-4f7a-96a4-acda07ead6d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:17.611220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33b868-23ce-4f7a-96a4-acda07ead6d7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:17.617380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:17.623816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf5b7d-8705-4591-8137-4b9a646e51e7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fa33b868-23ce-4f7a-96a4-acda07ead6d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:17.638105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:17.639669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:17.640569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:17.640569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:17.640569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:17.640569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 429/866] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:17.643946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33b868-23ce-4f7a-96a4-acda07ead6d7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:17.672315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268ebc3-887a-41d8-879b-813efe45e103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:17.673451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268ebc3-887a-41d8-879b-813efe45e103 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:17.673739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268ebc3-887a-41d8-879b-813efe45e103 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:17.674007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268ebc3-887a-41d8-879b-813efe45e103 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:17.674588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268ebc3-887a-41d8-879b-813efe45e103 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:17.676001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268ebc3-887a-41d8-879b-813efe45e103 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0268ebc3-887a-41d8-879b-813efe45e103 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:17.688402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:17.689334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:17.690225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 438/867] 10.222.0.86 () {56 vars in 1132 bytes} [Mon Aug 29 18:49:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:17.695984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268ebc3-887a-41d8-879b-813efe45e103 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:17.737096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:17.800499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33b868-23ce-4f7a-96a4-acda07ead6d7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:17.824835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268ebc3-887a-41d8-879b-813efe45e103 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:17.845908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebccee-0a9a-4e49-9603-316739e8fac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:17.846902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebccee-0a9a-4e49-9603-316739e8fac9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:17.847213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebccee-0a9a-4e49-9603-316739e8fac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:17.847490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebccee-0a9a-4e49-9603-316739e8fac9 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:17.848141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eebccee-0a9a-4e49-9603-316739e8fac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:17.848446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eebccee-0a9a-4e49-9603-316739e8fac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:17.850392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eebccee-0a9a-4e49-9603-316739e8fac9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:17.868321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933f312-a8a2-4091-9b99-eefa77bee31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:17.869090 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933f312-a8a2-4091-9b99-eefa77bee31c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:17.869442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933f312-a8a2-4091-9b99-eefa77bee31c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:17.869780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933f312-a8a2-4091-9b99-eefa77bee31c None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:17.870486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933f312-a8a2-4091-9b99-eefa77bee31c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:17.870858 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933f312-a8a2-4091-9b99-eefa77bee31c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:17.872376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933f312-a8a2-4091-9b99-eefa77bee31c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:17.873144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 430/868] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:17.875022 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebccee-0a9a-4e49-9603-316739e8fac9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:17.878472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 439/869] 10.222.0.86 () {56 vars in 1126 bytes} [Mon Aug 29 18:49:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:17.883704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933f312-a8a2-4091-9b99-eefa77bee31c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:17.923587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:17.999558 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebccee-0a9a-4e49-9603-316739e8fac9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:18.017690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933f312-a8a2-4091-9b99-eefa77bee31c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:18.019225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:18.045448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd5e0f-d606-49c0-a9ab-2d0dc3d1ed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:18.046741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd5e0f-d606-49c0-a9ab-2d0dc3d1ed81 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:18.047438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd5e0f-d606-49c0-a9ab-2d0dc3d1ed81 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:18.047812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd5e0f-d606-49c0-a9ab-2d0dc3d1ed81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:18.056273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd5e0f-d606-49c0-a9ab-2d0dc3d1ed81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:18.057209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06ea81-72e3-42cf-9315-2f03282a221e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:18.057819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd5e0f-d606-49c0-a9ab-2d0dc3d1ed81 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:18.057927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06ea81-72e3-42cf-9315-2f03282a221e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:18.058094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06ea81-72e3-42cf-9315-2f03282a221e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:18.058369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06ea81-72e3-42cf-9315-2f03282a221e None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:18.059196 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06ea81-72e3-42cf-9315-2f03282a221e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:18.059506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06ea81-72e3-42cf-9315-2f03282a221e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:18.061132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06ea81-72e3-42cf-9315-2f03282a221e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:18.066062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 440/870] 10.222.0.86 () {56 vars in 1126 bytes} [Mon Aug 29 18:49:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ebbd5e0f-d606-49c0-a9ab-2d0dc3d1ed81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:18.072241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:18.073200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:18.074113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:18.074113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:18.074113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:18.074113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:18.074113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:18.074113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:18.074113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:18.074113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:18.079181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 431/871] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:18.080408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06ea81-72e3-42cf-9315-2f03282a221e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:18.187759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:18.199239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06ea81-72e3-42cf-9315-2f03282a221e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:18.236213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3d5a7-67b3-4a50-872f-a9e2e08866d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:18.236827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3d5a7-67b3-4a50-872f-a9e2e08866d1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:18.237107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3d5a7-67b3-4a50-872f-a9e2e08866d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:18.237379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3d5a7-67b3-4a50-872f-a9e2e08866d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:18.238075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3d5a7-67b3-4a50-872f-a9e2e08866d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:18.238378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3d5a7-67b3-4a50-872f-a9e2e08866d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:18.239940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3d5a7-67b3-4a50-872f-a9e2e08866d1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:18.253599 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 441/872] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:18.259561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd5e0f-d606-49c0-a9ab-2d0dc3d1ed81 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:18.361864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd5e0f-d606-49c0-a9ab-2d0dc3d1ed81 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:18.400360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796686e-6b94-417f-8134-01e00e1b35f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:18.400975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796686e-6b94-417f-8134-01e00e1b35f0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:18.401256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796686e-6b94-417f-8134-01e00e1b35f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:18.401530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796686e-6b94-417f-8134-01e00e1b35f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:18.402256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796686e-6b94-417f-8134-01e00e1b35f0 None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=7634f750a07143a8b8d91cf0c8e78e16) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:18.402564 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796686e-6b94-417f-8134-01e00e1b35f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:18.404134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796686e-6b94-417f-8134-01e00e1b35f0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:18.410873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 432/873] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:18 2022] GET /identity/v3/role_assignments?user.id=7634f750a07143a8b8d91cf0c8e78e16&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:18.867983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f32ec-6dc3-4c1e-9ef1-83fc3a04ea4d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:18.868667 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f32ec-6dc3-4c1e-9ef1-83fc3a04ea4d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:18.868667 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f32ec-6dc3-4c1e-9ef1-83fc3a04ea4d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:18.869982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 442/874] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:18.876529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9848336-a82c-49dc-a0f1-68118b313bff None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:18.876529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9848336-a82c-49dc-a0f1-68118b313bff None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:18.876529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9848336-a82c-49dc-a0f1-68118b313bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:18.895484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e9848336-a82c-49dc-a0f1-68118b313bff None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:18.896999 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e9848336-a82c-49dc-a0f1-68118b313bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:18.927183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:18.966758 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 433/875] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:18 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:19.246166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f32ec-6dc3-4c1e-9ef1-83fc3a04ea4d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:19.358943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f32ec-6dc3-4c1e-9ef1-83fc3a04ea4d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:19.399145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38360100-945f-4e2f-97f8-01b86662a740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:19.399778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38360100-945f-4e2f-97f8-01b86662a740 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:19.400059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38360100-945f-4e2f-97f8-01b86662a740 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:19.400335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38360100-945f-4e2f-97f8-01b86662a740 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:19.400922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38360100-945f-4e2f-97f8-01b86662a740 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:19.402285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38360100-945f-4e2f-97f8-01b86662a740 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-38360100-945f-4e2f-97f8-01b86662a740 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:19.412546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:19.413591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:19.414439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 443/876] 10.222.0.86 () {56 vars in 1126 bytes} [Mon Aug 29 18:49:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:19.422448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9848336-a82c-49dc-a0f1-68118b313bff None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:19.525943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9848336-a82c-49dc-a0f1-68118b313bff None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:19.560661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598de71c-5824-4e24-8e73-69d3cae27f13 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:19.560976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598de71c-5824-4e24-8e73-69d3cae27f13 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:19.561212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51247ed4-d91e-4999-92c8-aabae021e3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:19.561347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598de71c-5824-4e24-8e73-69d3cae27f13 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:19.561888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51247ed4-d91e-4999-92c8-aabae021e3df None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:19.562170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51247ed4-d91e-4999-92c8-aabae021e3df None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:19.562440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51247ed4-d91e-4999-92c8-aabae021e3df None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:19.562544 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 444/877] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:19.563167 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51247ed4-d91e-4999-92c8-aabae021e3df None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:19.563469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51247ed4-d91e-4999-92c8-aabae021e3df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:19.564914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51247ed4-d91e-4999-92c8-aabae021e3df None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:19.567823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5140-ce03-49bf-a945-c07e6f53947f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:19.568111 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5140-ce03-49bf-a945-c07e6f53947f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:19.568386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5140-ce03-49bf-a945-c07e6f53947f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:19.570326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 434/878] 10.222.0.86 () {56 vars in 1122 bytes} [Mon Aug 29 18:49:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:19.576151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51247ed4-d91e-4999-92c8-aabae021e3df None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:19.595572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a10f5140-ce03-49bf-a945-c07e6f53947f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:19.596857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a10f5140-ce03-49bf-a945-c07e6f53947f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:19.609023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:19.610967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:19.692330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51247ed4-d91e-4999-92c8-aabae021e3df None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:19.697203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 445/879] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:19.732485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc29433-a581-4552-bb7c-7cbc4ebfc1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:19.733419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc29433-a581-4552-bb7c-7cbc4ebfc1fa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:19.733948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc29433-a581-4552-bb7c-7cbc4ebfc1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:19.734437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc29433-a581-4552-bb7c-7cbc4ebfc1fa None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:19.735293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc29433-a581-4552-bb7c-7cbc4ebfc1fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:19.735893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc29433-a581-4552-bb7c-7cbc4ebfc1fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:19.737431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc29433-a581-4552-bb7c-7cbc4ebfc1fa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:19.742732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 435/880] 10.222.0.86 () {56 vars in 1122 bytes} [Mon Aug 29 18:49:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:19.974166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f5140-ce03-49bf-a945-c07e6f53947f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:20.083583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f5140-ce03-49bf-a945-c07e6f53947f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:20.096438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41a5fd-d99e-4cde-846c-56c3c92b1a74 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:20.097384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41a5fd-d99e-4cde-846c-56c3c92b1a74 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:20.097860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41a5fd-d99e-4cde-846c-56c3c92b1a74 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:20.103219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 436/881] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:20.104922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743a6b-e91a-4b46-b269-80ac4e19c5f2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:20.105415 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743a6b-e91a-4b46-b269-80ac4e19c5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:20.105949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743a6b-e91a-4b46-b269-80ac4e19c5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:20.120750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42584a08-f1a7-423c-87aa-0021886eaf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:20.121927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42584a08-f1a7-423c-87aa-0021886eaf82 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:20.122209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42584a08-f1a7-423c-87aa-0021886eaf82 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:20.122477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42584a08-f1a7-423c-87aa-0021886eaf82 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:20.123089 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42584a08-f1a7-423c-87aa-0021886eaf82 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:20.124458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42584a08-f1a7-423c-87aa-0021886eaf82 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:20.128133 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-19743a6b-e91a-4b46-b269-80ac4e19c5f2 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:20.129575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-19743a6b-e91a-4b46-b269-80ac4e19c5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-42584a08-f1a7-423c-87aa-0021886eaf82 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:20.133578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:20.134576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:20.140119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 446/882] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:49:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:20.144654 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42584a08-f1a7-423c-87aa-0021886eaf82 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:20.160139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:20.227096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 437/883] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:20.277525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42584a08-f1a7-423c-87aa-0021886eaf82 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:20.313726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620dba6-bc32-460c-8656-f120cbde4902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:20.314331 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620dba6-bc32-460c-8656-f120cbde4902 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:20.314609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620dba6-bc32-460c-8656-f120cbde4902 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:20.314878 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620dba6-bc32-460c-8656-f120cbde4902 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:20.315534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620dba6-bc32-460c-8656-f120cbde4902 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:20.315834 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620dba6-bc32-460c-8656-f120cbde4902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:20.317198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620dba6-bc32-460c-8656-f120cbde4902 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:20.326768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 447/884] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:49:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:20.332882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19743a6b-e91a-4b46-b269-80ac4e19c5f2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:20.362111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:20.437328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19743a6b-e91a-4b46-b269-80ac4e19c5f2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:20.472673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabc45a-f9c5-468f-bbee-8211533ac47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:20.473611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabc45a-f9c5-468f-bbee-8211533ac47c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:20.474086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabc45a-f9c5-468f-bbee-8211533ac47c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:20.474576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabc45a-f9c5-468f-bbee-8211533ac47c None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:20.475496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabc45a-f9c5-468f-bbee-8211533ac47c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:20.475995 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabc45a-f9c5-468f-bbee-8211533ac47c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:20.477608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabc45a-f9c5-468f-bbee-8211533ac47c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:20.487229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 438/885] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:49:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:20.490497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620dba6-bc32-460c-8656-f120cbde4902 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:20.525893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabc45a-f9c5-468f-bbee-8211533ac47c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:20.571261 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:20.604894 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620dba6-bc32-460c-8656-f120cbde4902 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:20.628624 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:20.640586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabc45a-f9c5-468f-bbee-8211533ac47c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:20.640923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d9d33-77cc-4756-adf5-6e9c289ac40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:20.641533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d9d33-77cc-4756-adf5-6e9c289ac40f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:20.641813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d9d33-77cc-4756-adf5-6e9c289ac40f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:20.642090 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d9d33-77cc-4756-adf5-6e9c289ac40f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:20.642800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d9d33-77cc-4756-adf5-6e9c289ac40f None admin] RBAC: Adding query filter params (interface=public, service_id=5f3bd8f9bdee44cdada5c6f4477fd468, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:20.643121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d9d33-77cc-4756-adf5-6e9c289ac40f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:20.644487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d9d33-77cc-4756-adf5-6e9c289ac40f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:20.649395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 448/886] 10.222.0.86 () {56 vars in 1251 bytes} [Mon Aug 29 18:49:20 2022] GET /identity/v3/endpoints?service_id=5f3bd8f9bdee44cdada5c6f4477fd468&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:20.650542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38047a01-f60d-4516-a143-f225e74428da None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:20.650837 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38047a01-f60d-4516-a143-f225e74428da None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:20.651125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38047a01-f60d-4516-a143-f225e74428da None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:20.652460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 449/887] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:20.655623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38047a01-f60d-4516-a143-f225e74428da None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:20.681629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fec90-4b2c-44a5-aa71-a2f2784549df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:20.682532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fec90-4b2c-44a5-aa71-a2f2784549df None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:20.682926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fec90-4b2c-44a5-aa71-a2f2784549df None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:20.686597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fec90-4b2c-44a5-aa71-a2f2784549df None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:20.686597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fec90-4b2c-44a5-aa71-a2f2784549df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:20.686597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fec90-4b2c-44a5-aa71-a2f2784549df None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:20.702895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 439/888] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:49:20 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:20.704188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c377-41dc-40d5-b93f-b08843103fc9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:20.704488 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c377-41dc-40d5-b93f-b08843103fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:20.704762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c377-41dc-40d5-b93f-b08843103fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:20.740675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ddc3c377-41dc-40d5-b93f-b08843103fc9 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:20.742842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ddc3c377-41dc-40d5-b93f-b08843103fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:20.799791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38047a01-f60d-4516-a143-f225e74428da None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:20.838871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d659b-f7bc-4bd6-8901-37c5ea1b1d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:20.839807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d659b-f7bc-4bd6-8901-37c5ea1b1d5b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:20.840806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d659b-f7bc-4bd6-8901-37c5ea1b1d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:20.841180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d659b-f7bc-4bd6-8901-37c5ea1b1d5b None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:20.841969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d659b-f7bc-4bd6-8901-37c5ea1b1d5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:20.843817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d659b-f7bc-4bd6-8901-37c5ea1b1d5b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:20.851436 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 440/889] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:20.851631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:20.863197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 450/890] 10.222.0.86 () {56 vars in 1087 bytes} [Mon Aug 29 18:49:20 2022] GET /identity/v3/services => generated 1268 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 18:49:21.130623 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3c377-41dc-40d5-b93f-b08843103fc9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:21.217913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:21.240767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3c377-41dc-40d5-b93f-b08843103fc9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:21.277518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7151d2-7b35-4969-a360-145498994f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:21.278431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7151d2-7b35-4969-a360-145498994f49 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:21.278934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7151d2-7b35-4969-a360-145498994f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:21.279368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7151d2-7b35-4969-a360-145498994f49 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:21.290285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7151d2-7b35-4969-a360-145498994f49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ef7151d2-7b35-4969-a360-145498994f49 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:21.296062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:21.297228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:21.298261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:21.300609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 441/891] 10.222.0.86 () {56 vars in 1120 bytes} [Mon Aug 29 18:49:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:21.306629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d659b-f7bc-4bd6-8901-37c5ea1b1d5b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:21.414348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d659b-f7bc-4bd6-8901-37c5ea1b1d5b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:21.450304 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1888e-394f-4dcf-a4b3-b4fcbf55f8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:21.450915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1888e-394f-4dcf-a4b3-b4fcbf55f8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:21.451230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1888e-394f-4dcf-a4b3-b4fcbf55f8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:21.451504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1888e-394f-4dcf-a4b3-b4fcbf55f8b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:21.452247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1888e-394f-4dcf-a4b3-b4fcbf55f8b5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:21.452556 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1888e-394f-4dcf-a4b3-b4fcbf55f8b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:21.453927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1888e-394f-4dcf-a4b3-b4fcbf55f8b5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:21.462647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 451/892] 10.222.0.86 () {56 vars in 1115 bytes} [Mon Aug 29 18:49:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:21.468928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7151d2-7b35-4969-a360-145498994f49 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:21.572833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7151d2-7b35-4969-a360-145498994f49 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:21.608146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d1365-d0e3-4fc9-8397-45e55845f176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:21.609145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d1365-d0e3-4fc9-8397-45e55845f176 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:21.609642 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d1365-d0e3-4fc9-8397-45e55845f176 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:21.610111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d1365-d0e3-4fc9-8397-45e55845f176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:21.618868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:21.620895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d1365-d0e3-4fc9-8397-45e55845f176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:21.623565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d1365-d0e3-4fc9-8397-45e55845f176 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4d8d1365-d0e3-4fc9-8397-45e55845f176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:21.639466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:21.640432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:21.641339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:21.641339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:21.641339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:21.643459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 442/893] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:21.649462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1888e-394f-4dcf-a4b3-b4fcbf55f8b5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:21.703132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff61eb-026c-487d-920c-34a3e91d5d4a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:21.703422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff61eb-026c-487d-920c-34a3e91d5d4a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:21.703707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff61eb-026c-487d-920c-34a3e91d5d4a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:21.705306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 443/894] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:21.710190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e331d-79bc-455c-9bd3-f2a7e41d86c1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:21.710479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e331d-79bc-455c-9bd3-f2a7e41d86c1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:21.710759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e331d-79bc-455c-9bd3-f2a7e41d86c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:21.715036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:21.728971 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-af5e331d-79bc-455c-9bd3-f2a7e41d86c1 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:21.730179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-af5e331d-79bc-455c-9bd3-f2a7e41d86c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:21.768661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1888e-394f-4dcf-a4b3-b4fcbf55f8b5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:21.803422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe655bd-b3b9-411b-a3f5-5d3d14131f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:21.804036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe655bd-b3b9-411b-a3f5-5d3d14131f30 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:21.804317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe655bd-b3b9-411b-a3f5-5d3d14131f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:21.804592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe655bd-b3b9-411b-a3f5-5d3d14131f30 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:21.805174 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 444/895] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:21.811224 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe655bd-b3b9-411b-a3f5-5d3d14131f30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:21.811966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe655bd-b3b9-411b-a3f5-5d3d14131f30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:21.813634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe655bd-b3b9-411b-a3f5-5d3d14131f30 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:21.827977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 452/896] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:21.833925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e331d-79bc-455c-9bd3-f2a7e41d86c1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:21.941262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e331d-79bc-455c-9bd3-f2a7e41d86c1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:21.964764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:21.977481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be641d1f-76af-49c1-88ae-28de8aa037d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:21.978390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be641d1f-76af-49c1-88ae-28de8aa037d4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:21.978888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be641d1f-76af-49c1-88ae-28de8aa037d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:21.979380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be641d1f-76af-49c1-88ae-28de8aa037d4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:21.991460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be641d1f-76af-49c1-88ae-28de8aa037d4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:21.992908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be641d1f-76af-49c1-88ae-28de8aa037d4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-be641d1f-76af-49c1-88ae-28de8aa037d4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:22.005567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:22.006570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:22.007567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:22.007567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:22.007567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:22.007567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:22.007567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:22.007567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:22.007567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:22.007567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:22.007567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:22.007567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:22.007567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 445/897] 10.222.0.86 () {56 vars in 1117 bytes} [Mon Aug 29 18:49:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:22.014707 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe655bd-b3b9-411b-a3f5-5d3d14131f30 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:22.079177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be641d1f-76af-49c1-88ae-28de8aa037d4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:22.125786 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe655bd-b3b9-411b-a3f5-5d3d14131f30 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:22.146981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:22.170249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e844a-a02a-4f90-ac17-1e43324c80ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:22.171576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e844a-a02a-4f90-ac17-1e43324c80ad None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:22.172030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e844a-a02a-4f90-ac17-1e43324c80ad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:22.172492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e844a-a02a-4f90-ac17-1e43324c80ad None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:22.173401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e844a-a02a-4f90-ac17-1e43324c80ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:22.173893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e844a-a02a-4f90-ac17-1e43324c80ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:22.175638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e844a-a02a-4f90-ac17-1e43324c80ad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:22.188988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be641d1f-76af-49c1-88ae-28de8aa037d4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:22.201726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 453/898] 10.222.0.86 () {56 vars in 1149 bytes} [Mon Aug 29 18:49:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:22.207400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e844a-a02a-4f90-ac17-1e43324c80ad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:22.234446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c5f13-0bd5-489f-b3d3-37aff7c34c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:22.235149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c5f13-0bd5-489f-b3d3-37aff7c34c15 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:22.235427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c5f13-0bd5-489f-b3d3-37aff7c34c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:22.235734 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c5f13-0bd5-489f-b3d3-37aff7c34c15 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:22.236368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c5f13-0bd5-489f-b3d3-37aff7c34c15 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:22.237721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c5f13-0bd5-489f-b3d3-37aff7c34c15 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:22.256167 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 446/899] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:49:22 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:49:22.338987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e844a-a02a-4f90-ac17-1e43324c80ad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:22.386738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be926d1b-cb7d-4184-924e-7d3ae75942ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:22.388298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be926d1b-cb7d-4184-924e-7d3ae75942ee None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:22.388828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be926d1b-cb7d-4184-924e-7d3ae75942ee None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:22.389299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be926d1b-cb7d-4184-924e-7d3ae75942ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:22.439148 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:22.478496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be926d1b-cb7d-4184-924e-7d3ae75942ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:22.478496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be926d1b-cb7d-4184-924e-7d3ae75942ee None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:22.582968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 454/900] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:22 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:49:22.588641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c5f13-0bd5-489f-b3d3-37aff7c34c15 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:22.634516 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:22.698448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c5f13-0bd5-489f-b3d3-37aff7c34c15 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:22.734145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7f51a-3c57-4909-a27c-44cb5fd67474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:22.734703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f51a-3c57-4909-a27c-44cb5fd67474 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:22.734986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f51a-3c57-4909-a27c-44cb5fd67474 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:22.735284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f51a-3c57-4909-a27c-44cb5fd67474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:22.746236 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7f51a-3c57-4909-a27c-44cb5fd67474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:22.747774 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7f51a-3c57-4909-a27c-44cb5fd67474 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0ae7f51a-3c57-4909-a27c-44cb5fd67474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:22.763425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:22.764419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:22.765369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:22.765369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:22.765369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:22.765369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 447/901] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:22.771891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be926d1b-cb7d-4184-924e-7d3ae75942ee None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:22.790333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca52323-6388-49eb-8a09-e097b81851c1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:22.790639 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca52323-6388-49eb-8a09-e097b81851c1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:22.791256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca52323-6388-49eb-8a09-e097b81851c1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:22.792919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 448/902] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:22.799210 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe716f97-db7a-4258-bbed-5e82257850ad None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:22.799711 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe716f97-db7a-4258-bbed-5e82257850ad None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:22.800149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe716f97-db7a-4258-bbed-5e82257850ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:22.820141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fe716f97-db7a-4258-bbed-5e82257850ad None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:22.821368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fe716f97-db7a-4258-bbed-5e82257850ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:22.862689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:22.898036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 449/903] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:22.907739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be926d1b-cb7d-4184-924e-7d3ae75942ee None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:22.917100 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065534f8-7fac-4d31-980e-4fe0d050873e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:22.917392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065534f8-7fac-4d31-980e-4fe0d050873e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:22.917672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065534f8-7fac-4d31-980e-4fe0d050873e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:22.922870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 450/904] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:22.924087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18402835-50b8-42bf-b4af-c017bf0ea170 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:22.924378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18402835-50b8-42bf-b4af-c017bf0ea170 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:22.924660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18402835-50b8-42bf-b4af-c017bf0ea170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:22.943768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-18402835-50b8-42bf-b4af-c017bf0ea170 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:22.945579 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-18402835-50b8-42bf-b4af-c017bf0ea170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:22.946316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbabed-45bb-45c4-9876-a728729963b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:22.946928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbabed-45bb-45c4-9876-a728729963b7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:22.947238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbabed-45bb-45c4-9876-a728729963b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:22.947511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbabed-45bb-45c4-9876-a728729963b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:22.948151 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbabed-45bb-45c4-9876-a728729963b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:22.948451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbabed-45bb-45c4-9876-a728729963b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:22.950381 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbabed-45bb-45c4-9876-a728729963b7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:22.962205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 455/905] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:22.968109 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbabed-45bb-45c4-9876-a728729963b7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:22.970004 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:23.040498 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 451/906] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:23.078648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbabed-45bb-45c4-9876-a728729963b7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:23.113160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efb005-8cc5-42cd-924d-72aaff4a2a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:23.113821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efb005-8cc5-42cd-924d-72aaff4a2a3f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:23.114116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efb005-8cc5-42cd-924d-72aaff4a2a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:23.114396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efb005-8cc5-42cd-924d-72aaff4a2a3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:23.115219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:23.122602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efb005-8cc5-42cd-924d-72aaff4a2a3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:23.124259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efb005-8cc5-42cd-924d-72aaff4a2a3f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e4efb005-8cc5-42cd-924d-72aaff4a2a3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:23.136207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:23.138104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:23.140243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:23.140243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:23.140243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:23.140243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:23.140243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:23.140243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:23.140243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:23.140243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:23.140243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 456/907] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:23.143900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18402835-50b8-42bf-b4af-c017bf0ea170 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:23.216589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efb005-8cc5-42cd-924d-72aaff4a2a3f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:23.248603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18402835-50b8-42bf-b4af-c017bf0ea170 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:23.296035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b70789-236b-43e9-a54a-c901c06f4e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:23.296633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b70789-236b-43e9-a54a-c901c06f4e57 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:23.296917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b70789-236b-43e9-a54a-c901c06f4e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:23.297189 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b70789-236b-43e9-a54a-c901c06f4e57 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:23.298138 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b70789-236b-43e9-a54a-c901c06f4e57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:23.298442 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b70789-236b-43e9-a54a-c901c06f4e57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:23.300884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b70789-236b-43e9-a54a-c901c06f4e57 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:23.319233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 452/908] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:23.322766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b70789-236b-43e9-a54a-c901c06f4e57 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:23.340129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:23.345175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efb005-8cc5-42cd-924d-72aaff4a2a3f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:23.380985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cea92a-05ff-4d89-a80b-0642a1ca42b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:23.381619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cea92a-05ff-4d89-a80b-0642a1ca42b0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:23.381900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cea92a-05ff-4d89-a80b-0642a1ca42b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:23.382170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cea92a-05ff-4d89-a80b-0642a1ca42b0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:23.382749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cea92a-05ff-4d89-a80b-0642a1ca42b0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:23.384156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cea92a-05ff-4d89-a80b-0642a1ca42b0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-56cea92a-05ff-4d89-a80b-0642a1ca42b0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:23.393421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:23.394446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:23.395396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 457/909] 10.222.0.86 () {56 vars in 1132 bytes} [Mon Aug 29 18:49:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:23.396205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cea92a-05ff-4d89-a80b-0642a1ca42b0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:23.437538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b70789-236b-43e9-a54a-c901c06f4e57 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:23.463534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:23.474565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3d633-2513-4525-9dc5-7a12186b950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:23.475229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3d633-2513-4525-9dc5-7a12186b950e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:23.475514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3d633-2513-4525-9dc5-7a12186b950e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:23.475786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3d633-2513-4525-9dc5-7a12186b950e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:23.476538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3d633-2513-4525-9dc5-7a12186b950e None admin] RBAC: Adding query filter params (role.id=4ee6c8fb1f294eb0824448e1f26984d6, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=7634f750a07143a8b8d91cf0c8e78e16) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:23.476849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3d633-2513-4525-9dc5-7a12186b950e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:23.478326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3d633-2513-4525-9dc5-7a12186b950e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:23.495045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 453/910] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:23 2022] GET /identity/v3/role_assignments?user.id=7634f750a07143a8b8d91cf0c8e78e16&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=4ee6c8fb1f294eb0824448e1f26984d6 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:23.495045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3d633-2513-4525-9dc5-7a12186b950e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:23.528347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cea92a-05ff-4d89-a80b-0642a1ca42b0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:23.566579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e951f07-bd4b-4c6d-8e91-ced189c88b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:23.567559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951f07-bd4b-4c6d-8e91-ced189c88b1c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:23.567950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951f07-bd4b-4c6d-8e91-ced189c88b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:23.568321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951f07-bd4b-4c6d-8e91-ced189c88b1c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:23.569275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e951f07-bd4b-4c6d-8e91-ced189c88b1c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:23.570945 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e951f07-bd4b-4c6d-8e91-ced189c88b1c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7e951f07-bd4b-4c6d-8e91-ced189c88b1c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:23.581653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:23.582794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:49:23.583655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:23.588510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 458/911] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:49:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:23.589531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e951f07-bd4b-4c6d-8e91-ced189c88b1c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:23.594284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:23.623818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3d633-2513-4525-9dc5-7a12186b950e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:23.662682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3f1cc-b4e8-4e27-b172-8076a8d4cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:23.663333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f1cc-b4e8-4e27-b172-8076a8d4cb89 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:23.663616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f1cc-b4e8-4e27-b172-8076a8d4cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:23.663894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f1cc-b4e8-4e27-b172-8076a8d4cb89 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:23.664526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3f1cc-b4e8-4e27-b172-8076a8d4cb89 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:23.664833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3f1cc-b4e8-4e27-b172-8076a8d4cb89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:23.668325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3f1cc-b4e8-4e27-b172-8076a8d4cb89 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:23.673496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 454/912] 10.222.0.86 () {56 vars in 1126 bytes} [Mon Aug 29 18:49:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:23.681057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3f1cc-b4e8-4e27-b172-8076a8d4cb89 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:23.717172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e951f07-bd4b-4c6d-8e91-ced189c88b1c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:23.739066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:23.755190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa872e-11df-4824-baab-2537dd5720a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:23.755900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa872e-11df-4824-baab-2537dd5720a6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:23.756250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa872e-11df-4824-baab-2537dd5720a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:23.756686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa872e-11df-4824-baab-2537dd5720a6 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:23.757410 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa872e-11df-4824-baab-2537dd5720a6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:23.757962 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa872e-11df-4824-baab-2537dd5720a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:23.759435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa872e-11df-4824-baab-2537dd5720a6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:23.770287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 459/913] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:49:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:23.771362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa872e-11df-4824-baab-2537dd5720a6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:23.795598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3f1cc-b4e8-4e27-b172-8076a8d4cb89 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:23.834006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb08623-5f88-4adf-9767-2aed05cfd723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:23.834602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb08623-5f88-4adf-9767-2aed05cfd723 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:23.834893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb08623-5f88-4adf-9767-2aed05cfd723 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:23.835324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb08623-5f88-4adf-9767-2aed05cfd723 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:23.835916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb08623-5f88-4adf-9767-2aed05cfd723 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:23.836364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb08623-5f88-4adf-9767-2aed05cfd723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:23.837990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb08623-5f88-4adf-9767-2aed05cfd723 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:23.843978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 455/914] 10.222.0.86 () {56 vars in 1126 bytes} [Mon Aug 29 18:49:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:23.853237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb08623-5f88-4adf-9767-2aed05cfd723 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:23.864672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:23.894595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa872e-11df-4824-baab-2537dd5720a6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:23.937282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd2aca-d908-4726-ab64-406089c98156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:23.938052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd2aca-d908-4726-ab64-406089c98156 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:23.938400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd2aca-d908-4726-ab64-406089c98156 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:23.938745 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd2aca-d908-4726-ab64-406089c98156 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:23.939627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd2aca-d908-4726-ab64-406089c98156 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:23.939999 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd2aca-d908-4726-ab64-406089c98156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:23.941949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd2aca-d908-4726-ab64-406089c98156 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:23.948704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 460/915] 10.222.0.86 () {56 vars in 1114 bytes} [Mon Aug 29 18:49:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:23.955099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd2aca-d908-4726-ab64-406089c98156 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:23.986876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb08623-5f88-4adf-9767-2aed05cfd723 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:24.017968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:24.034395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefbd0f-c861-4d6c-bab5-83a682c79b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:24.035202 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefbd0f-c861-4d6c-bab5-83a682c79b4c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:24.035586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefbd0f-c861-4d6c-bab5-83a682c79b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:24.035963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefbd0f-c861-4d6c-bab5-83a682c79b4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:24.036862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefbd0f-c861-4d6c-bab5-83a682c79b4c None admin] RBAC: Adding query filter params (interface=public, service_id=bf8ab610f4504bca89f3c67414017c08, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:24.037285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefbd0f-c861-4d6c-bab5-83a682c79b4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:24.039468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefbd0f-c861-4d6c-bab5-83a682c79b4c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:24.051077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 456/916] 10.222.0.86 () {56 vars in 1251 bytes} [Mon Aug 29 18:49:23 2022] GET /identity/v3/endpoints?service_id=bf8ab610f4504bca89f3c67414017c08&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:24.052441 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefbd0f-c861-4d6c-bab5-83a682c79b4c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:24.080773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd2aca-d908-4726-ab64-406089c98156 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:24.117781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9da918-1c0f-4495-996f-566782261a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:24.118319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9da918-1c0f-4495-996f-566782261a22 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:24.118650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9da918-1c0f-4495-996f-566782261a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:24.118926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9da918-1c0f-4495-996f-566782261a22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:24.119699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9da918-1c0f-4495-996f-566782261a22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:24.121294 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9da918-1c0f-4495-996f-566782261a22 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:24.151243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 461/917] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 18:49:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:24.159452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:24.203449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefbd0f-c861-4d6c-bab5-83a682c79b4c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:24.255823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff02f32-6fcf-4bd1-b0bd-a92c51bcfb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:24.256450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff02f32-6fcf-4bd1-b0bd-a92c51bcfb90 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:24.256705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff02f32-6fcf-4bd1-b0bd-a92c51bcfb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:24.256971 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff02f32-6fcf-4bd1-b0bd-a92c51bcfb90 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:24.257573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff02f32-6fcf-4bd1-b0bd-a92c51bcfb90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:24.259129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff02f32-6fcf-4bd1-b0bd-a92c51bcfb90 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:24.266222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 457/918] 10.222.0.86 () {56 vars in 1087 bytes} [Mon Aug 29 18:49:24 2022] GET /identity/v3/services => generated 1497 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:49:24.363372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461551-4e0c-444e-adb7-6a85fc374b43 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:24.363893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461551-4e0c-444e-adb7-6a85fc374b43 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:24.363991 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461551-4e0c-444e-adb7-6a85fc374b43 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:24.365255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 462/919] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:24.371151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589dabe-32b6-4c3a-9aa2-ed3df50b8216 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:24.371972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589dabe-32b6-4c3a-9aa2-ed3df50b8216 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:24.372207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589dabe-32b6-4c3a-9aa2-ed3df50b8216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:24.392261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8589dabe-32b6-4c3a-9aa2-ed3df50b8216 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:24.393479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8589dabe-32b6-4c3a-9aa2-ed3df50b8216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:24.468486 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 458/920] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:24.747344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4461551-4e0c-444e-adb7-6a85fc374b43 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:24.856539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4461551-4e0c-444e-adb7-6a85fc374b43 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:24.859229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:24.895012 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e9cf3-923b-4ee4-8986-163ca8df1c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:24.895661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e9cf3-923b-4ee4-8986-163ca8df1c40 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:24.895939 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e9cf3-923b-4ee4-8986-163ca8df1c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:24.896214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e9cf3-923b-4ee4-8986-163ca8df1c40 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:24.896798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e9cf3-923b-4ee4-8986-163ca8df1c40 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:24.898157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e9cf3-923b-4ee4-8986-163ca8df1c40 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5a1e9cf3-923b-4ee4-8986-163ca8df1c40 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:24.907726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:24.908689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:24.909663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 463/921] 10.222.0.86 () {56 vars in 1126 bytes} [Mon Aug 29 18:49:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:24.914919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589dabe-32b6-4c3a-9aa2-ed3df50b8216 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:24.978230 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:25.021338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589dabe-32b6-4c3a-9aa2-ed3df50b8216 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:25.056669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984b7f1-1ec6-41d0-b975-87b3aac164c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:25.057292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b7f1-1ec6-41d0-b975-87b3aac164c2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:25.057615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b7f1-1ec6-41d0-b975-87b3aac164c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:25.057894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b7f1-1ec6-41d0-b975-87b3aac164c2 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:25.058523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984b7f1-1ec6-41d0-b975-87b3aac164c2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:25.059518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984b7f1-1ec6-41d0-b975-87b3aac164c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:25.060903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984b7f1-1ec6-41d0-b975-87b3aac164c2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:25.066269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 459/922] 10.222.0.86 () {56 vars in 1122 bytes} [Mon Aug 29 18:49:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:25.072175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e9cf3-923b-4ee4-8986-163ca8df1c40 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:25.177517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e9cf3-923b-4ee4-8986-163ca8df1c40 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:25.218356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b10d6-e8b7-46e6-8d55-1a20f0184e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:25.218972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b10d6-e8b7-46e6-8d55-1a20f0184e1e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:25.219274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b10d6-e8b7-46e6-8d55-1a20f0184e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:25.219602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b10d6-e8b7-46e6-8d55-1a20f0184e1e None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:25.220248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b10d6-e8b7-46e6-8d55-1a20f0184e1e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:25.220553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b10d6-e8b7-46e6-8d55-1a20f0184e1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:25.221922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b10d6-e8b7-46e6-8d55-1a20f0184e1e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:25.227467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 464/923] 10.222.0.86 () {56 vars in 1122 bytes} [Mon Aug 29 18:49:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:25.233456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984b7f1-1ec6-41d0-b975-87b3aac164c2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:25.348101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984b7f1-1ec6-41d0-b975-87b3aac164c2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:25.379397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:25.384358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71802afd-a996-4373-b8c4-9be3f2bb6143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:25.384978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71802afd-a996-4373-b8c4-9be3f2bb6143 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:25.385262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71802afd-a996-4373-b8c4-9be3f2bb6143 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:25.385559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71802afd-a996-4373-b8c4-9be3f2bb6143 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:25.386268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71802afd-a996-4373-b8c4-9be3f2bb6143 None admin] RBAC: Adding query filter params (interface=public, service_id=f70e89ed0a9648b0a8ba8dcc3124d903, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:25.386586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71802afd-a996-4373-b8c4-9be3f2bb6143 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:25.388323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71802afd-a996-4373-b8c4-9be3f2bb6143 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:25.394464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 460/924] 10.222.0.86 () {56 vars in 1251 bytes} [Mon Aug 29 18:49:25 2022] GET /identity/v3/endpoints?service_id=f70e89ed0a9648b0a8ba8dcc3124d903&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:25.399985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b10d6-e8b7-46e6-8d55-1a20f0184e1e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:25.433390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:25.460263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff13f08-656f-4b5f-9f24-3c3369171472 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:25.460583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff13f08-656f-4b5f-9f24-3c3369171472 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:25.460822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff13f08-656f-4b5f-9f24-3c3369171472 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:25.462392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 461/925] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:25.467815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399fe89-2b25-4b52-b98d-319cb08a390a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:25.468104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399fe89-2b25-4b52-b98d-319cb08a390a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:25.468383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399fe89-2b25-4b52-b98d-319cb08a390a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:25.491679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5399fe89-2b25-4b52-b98d-319cb08a390a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:25.492799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5399fe89-2b25-4b52-b98d-319cb08a390a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:25.512073 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b10d6-e8b7-46e6-8d55-1a20f0184e1e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:25.557595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f7bb8-12bf-4447-95b6-f9bcb214e9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:25.558220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f7bb8-12bf-4447-95b6-f9bcb214e9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:25.558504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f7bb8-12bf-4447-95b6-f9bcb214e9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:25.558811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f7bb8-12bf-4447-95b6-f9bcb214e9e0 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:25.559462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f7bb8-12bf-4447-95b6-f9bcb214e9e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:25.560839 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f7bb8-12bf-4447-95b6-f9bcb214e9e0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:25.570478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 465/926] 10.222.0.86 () {56 vars in 1087 bytes} [Mon Aug 29 18:49:25 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:49:25.583156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 462/927] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:25.851268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f7bb8-12bf-4447-95b6-f9bcb214e9e0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:25.958774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:25.965606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f7bb8-12bf-4447-95b6-f9bcb214e9e0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:26.001084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcceec-14ea-4e03-bb15-14e2c2b43ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:26.001732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcceec-14ea-4e03-bb15-14e2c2b43ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.002015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcceec-14ea-4e03-bb15-14e2c2b43ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.002290 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcceec-14ea-4e03-bb15-14e2c2b43ba2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:26.012859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcceec-14ea-4e03-bb15-14e2c2b43ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-debcceec-14ea-4e03-bb15-14e2c2b43ba2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:26.018972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:26.020157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:26.021189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:26.023823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 466/928] 10.222.0.86 () {56 vars in 1120 bytes} [Mon Aug 29 18:49:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:26.030203 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399fe89-2b25-4b52-b98d-319cb08a390a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:26.141317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399fe89-2b25-4b52-b98d-319cb08a390a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:26.159955 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:26.176068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667fbb4-a4f5-4489-8dd3-453d87e770d7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.176362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667fbb4-a4f5-4489-8dd3-453d87e770d7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.176601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667fbb4-a4f5-4489-8dd3-453d87e770d7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:26.178092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 467/929] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:26.183035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893d3d-fb44-4d03-8c33-d32ad523a333 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.183305 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893d3d-fb44-4d03-8c33-d32ad523a333 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.183580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893d3d-fb44-4d03-8c33-d32ad523a333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:26.185332 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a3b32-af94-41dd-904c-3c4a459e1aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:26.186250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a3b32-af94-41dd-904c-3c4a459e1aae None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.186844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a3b32-af94-41dd-904c-3c4a459e1aae None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.187328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a3b32-af94-41dd-904c-3c4a459e1aae None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:26.188278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a3b32-af94-41dd-904c-3c4a459e1aae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:26.188793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a3b32-af94-41dd-904c-3c4a459e1aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:26.190437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a3b32-af94-41dd-904c-3c4a459e1aae None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:26.200223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 463/930] 10.222.0.86 () {56 vars in 1115 bytes} [Mon Aug 29 18:49:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:26.206963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a3b32-af94-41dd-904c-3c4a459e1aae None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:26.212928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-28893d3d-fb44-4d03-8c33-d32ad523a333 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:26.214433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-28893d3d-fb44-4d03-8c33-d32ad523a333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:26.299224 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 468/931] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:26.318563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a3b32-af94-41dd-904c-3c4a459e1aae None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:26.353171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5aacb-c79b-480f-905d-6e607c97bbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:26.354547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5aacb-c79b-480f-905d-6e607c97bbe6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.355043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5aacb-c79b-480f-905d-6e607c97bbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.355528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5aacb-c79b-480f-905d-6e607c97bbe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:26.366941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5aacb-c79b-480f-905d-6e607c97bbe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:26.368667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5aacb-c79b-480f-905d-6e607c97bbe6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fbf5aacb-c79b-480f-905d-6e607c97bbe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:26.383489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:26.384460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:26.385345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:26.385345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:26.385345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:26.387446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 464/932] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:26.399114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28893d3d-fb44-4d03-8c33-d32ad523a333 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:26.428789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed478e66-9f28-4d5d-b85d-39861fa4fb29 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.429080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed478e66-9f28-4d5d-b85d-39861fa4fb29 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.429527 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed478e66-9f28-4d5d-b85d-39861fa4fb29 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:26.431994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 465/933] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:26.440328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87564e-e66e-455d-92ac-993ff988d6db None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.440719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87564e-e66e-455d-92ac-993ff988d6db None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.441102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87564e-e66e-455d-92ac-993ff988d6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:26.460203 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fa87564e-e66e-455d-92ac-993ff988d6db None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:26.461391 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fa87564e-e66e-455d-92ac-993ff988d6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:26.484879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:26.492988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:26.501393 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28893d3d-fb44-4d03-8c33-d32ad523a333 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:26.537201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665d32-4956-40d8-a99a-875d7dc6620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:26.537810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665d32-4956-40d8-a99a-875d7dc6620c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.538091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665d32-4956-40d8-a99a-875d7dc6620c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.538366 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665d32-4956-40d8-a99a-875d7dc6620c None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:26.539022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09665d32-4956-40d8-a99a-875d7dc6620c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:26.539349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09665d32-4956-40d8-a99a-875d7dc6620c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:26.540714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09665d32-4956-40d8-a99a-875d7dc6620c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:26.553184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 466/934] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:26.555637 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 469/935] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:26.562099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87564e-e66e-455d-92ac-993ff988d6db None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:26.600082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665d32-4956-40d8-a99a-875d7dc6620c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:26.695671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87564e-e66e-455d-92ac-993ff988d6db None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:26.733684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276c7d7-8e79-4a98-ad0e-96c40c31aa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:26.734654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276c7d7-8e79-4a98-ad0e-96c40c31aa2d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.735161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276c7d7-8e79-4a98-ad0e-96c40c31aa2d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.735662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276c7d7-8e79-4a98-ad0e-96c40c31aa2d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:26.743153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276c7d7-8e79-4a98-ad0e-96c40c31aa2d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:26.744843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276c7d7-8e79-4a98-ad0e-96c40c31aa2d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:26.749868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665d32-4956-40d8-a99a-875d7dc6620c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9276c7d7-8e79-4a98-ad0e-96c40c31aa2d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:26.757254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:26.758194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:26.759151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:26.759151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:26.759151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:26.759151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:26.759151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:26.759151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:26.759151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:26.759151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:26.759151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:26.759151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:26.761730 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 467/936] 10.222.0.86 () {56 vars in 1117 bytes} [Mon Aug 29 18:49:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:26.771122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276c7d7-8e79-4a98-ad0e-96c40c31aa2d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:26.789950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda2258-90c7-41fb-975b-8a5eae0ee425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:26.790557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2258-90c7-41fb-975b-8a5eae0ee425 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.790834 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2258-90c7-41fb-975b-8a5eae0ee425 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.791130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2258-90c7-41fb-975b-8a5eae0ee425 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:26.791775 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda2258-90c7-41fb-975b-8a5eae0ee425 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:26.793954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda2258-90c7-41fb-975b-8a5eae0ee425 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:26.796668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: WARNING keystone.server.flask.common [None req-6dda2258-90c7-41fb-975b-8a5eae0ee425 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 18:49:26.804409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:26.819929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.sql.core [None req-6dda2258-90c7-41fb-975b-8a5eae0ee425 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 18:49:26.819929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 18:49:26.819929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [parameters: {'id': '7a4253505f2746f085d3a9d0cad0a3b6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 18:49:26.819929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6dda2258-90c7-41fb-975b-8a5eae0ee425 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7a4253505f2746f085d3a9d0cad0a3b6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7a4253505f2746f085d3a9d0cad0a3b6. Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:49:26.837645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 29 18:49:26.838540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application yield resource Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise self.value Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application session.commit() Aug 29 18:49:26.839525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.flush() Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application compat.raise_( Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 18:49:26.841113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 18:49:26.842139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:49:26.842946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [parameters: {'id': '7a4253505f2746f085d3a9d0cad0a3b6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:26.845064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:26.847194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:26.849442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:26.849442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 29 18:49:26.849442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 29 18:49:26.849442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:26.849442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:26.849442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 29 18:49:26.849442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 29 18:49:26.849442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7a4253505f2746f085d3a9d0cad0a3b6. Aug 29 18:49:26.849442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:26.849442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 470/937] 10.222.0.86 () {60 vars in 1140 bytes} [Mon Aug 29 18:49:26 2022] POST /identity/v3/projects => generated 262 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:26.849442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda2258-90c7-41fb-975b-8a5eae0ee425 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:26.851709 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:26.901524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276c7d7-8e79-4a98-ad0e-96c40c31aa2d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:26.938404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbcd92-c9a7-43aa-ad7b-498439d3b15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:26.939153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbcd92-c9a7-43aa-ad7b-498439d3b15b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.939433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbcd92-c9a7-43aa-ad7b-498439d3b15b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.939705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbcd92-c9a7-43aa-ad7b-498439d3b15b None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:26.940531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbcd92-c9a7-43aa-ad7b-498439d3b15b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:26.940790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbcd92-c9a7-43aa-ad7b-498439d3b15b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:26.942308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbcd92-c9a7-43aa-ad7b-498439d3b15b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:26.955894 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda2258-90c7-41fb-975b-8a5eae0ee425 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:26.964777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 468/938] 10.222.0.86 () {56 vars in 1149 bytes} [Mon Aug 29 18:49:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:26.964777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbcd92-c9a7-43aa-ad7b-498439d3b15b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:26.994389 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9185625-1710-42c3-8ec7-c013e1227424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:26.995066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9185625-1710-42c3-8ec7-c013e1227424 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:26.995350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9185625-1710-42c3-8ec7-c013e1227424 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:26.995621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9185625-1710-42c3-8ec7-c013e1227424 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.002376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9185625-1710-42c3-8ec7-c013e1227424 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:27.004060 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9185625-1710-42c3-8ec7-c013e1227424 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e9185625-1710-42c3-8ec7-c013e1227424 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:27.015321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:27.016528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:27.017953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:27.017953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:27.017953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:27.017953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:27.017953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:27.017953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:27.017953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:27.017953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:27.021238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 471/939] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:27.022325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9185625-1710-42c3-8ec7-c013e1227424 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:27.051958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:27.077720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbcd92-c9a7-43aa-ad7b-498439d3b15b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:27.094348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:27.132449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca5f35-4b1e-4dd5-91b8-b42e64a613e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:27.133538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca5f35-4b1e-4dd5-91b8-b42e64a613e6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:27.133948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca5f35-4b1e-4dd5-91b8-b42e64a613e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:27.134374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca5f35-4b1e-4dd5-91b8-b42e64a613e6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.135343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca5f35-4b1e-4dd5-91b8-b42e64a613e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:27.136500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9185625-1710-42c3-8ec7-c013e1227424 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:27.137902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca5f35-4b1e-4dd5-91b8-b42e64a613e6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c8ca5f35-4b1e-4dd5-91b8-b42e64a613e6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:27.152329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:27.154095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:27.155182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 469/940] 10.222.0.86 () {56 vars in 1132 bytes} [Mon Aug 29 18:49:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:27.156133 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca5f35-4b1e-4dd5-91b8-b42e64a613e6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:27.180084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce063c83-9454-4eea-936e-b9672fb24530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:27.180981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce063c83-9454-4eea-936e-b9672fb24530 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:27.181257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce063c83-9454-4eea-936e-b9672fb24530 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:27.181531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce063c83-9454-4eea-936e-b9672fb24530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.195630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce063c83-9454-4eea-936e-b9672fb24530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:27.197400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce063c83-9454-4eea-936e-b9672fb24530 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ce063c83-9454-4eea-936e-b9672fb24530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:27.217805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:27.218969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:27.225017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:27.225017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:27.225017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:27.225017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 472/941] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:27.225017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce063c83-9454-4eea-936e-b9672fb24530 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:27.260812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca5f35-4b1e-4dd5-91b8-b42e64a613e6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:27.301408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c4dde-3bd5-46c4-8b98-245b6714a18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:27.302029 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c4dde-3bd5-46c4-8b98-245b6714a18b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:27.302308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c4dde-3bd5-46c4-8b98-245b6714a18b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:27.302581 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c4dde-3bd5-46c4-8b98-245b6714a18b None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.303280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c4dde-3bd5-46c4-8b98-245b6714a18b None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:27.303583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c4dde-3bd5-46c4-8b98-245b6714a18b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:27.305573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c4dde-3bd5-46c4-8b98-245b6714a18b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:27.339030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 470/942] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:27.339030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c4dde-3bd5-46c4-8b98-245b6714a18b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:27.339323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce063c83-9454-4eea-936e-b9672fb24530 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:27.342718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:27.347942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:27.373941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7bdd-3e87-4155-b023-d056cb229d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:27.374620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7bdd-3e87-4155-b023-d056cb229d46 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:27.374907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7bdd-3e87-4155-b023-d056cb229d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:27.375211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7bdd-3e87-4155-b023-d056cb229d46 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.375908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e7bdd-3e87-4155-b023-d056cb229d46 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:27.376214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e7bdd-3e87-4155-b023-d056cb229d46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:27.377903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e7bdd-3e87-4155-b023-d056cb229d46 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:27.387558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 473/943] 10.222.0.86 () {56 vars in 1126 bytes} [Mon Aug 29 18:49:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:27.399030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7bdd-3e87-4155-b023-d056cb229d46 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:27.468859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c4dde-3bd5-46c4-8b98-245b6714a18b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:27.513590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4550ee-e44a-46fe-bc62-552622e7cd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:27.514317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4550ee-e44a-46fe-bc62-552622e7cd32 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:27.514605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4550ee-e44a-46fe-bc62-552622e7cd32 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:27.514884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4550ee-e44a-46fe-bc62-552622e7cd32 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.515578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4550ee-e44a-46fe-bc62-552622e7cd32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:27.515887 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4550ee-e44a-46fe-bc62-552622e7cd32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:27.517291 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4550ee-e44a-46fe-bc62-552622e7cd32 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:27.526277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7bdd-3e87-4155-b023-d056cb229d46 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:27.531283 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 471/944] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:27.536675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4550ee-e44a-46fe-bc62-552622e7cd32 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:27.569306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e7378-c2bc-4697-8a15-a268b6321eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:27.570298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e7378-c2bc-4697-8a15-a268b6321eec None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:27.571354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e7378-c2bc-4697-8a15-a268b6321eec None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:27.571354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e7378-c2bc-4697-8a15-a268b6321eec None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.572284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e7378-c2bc-4697-8a15-a268b6321eec None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:27.572810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e7378-c2bc-4697-8a15-a268b6321eec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:27.575670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e7378-c2bc-4697-8a15-a268b6321eec None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:27.585372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 474/945] 10.222.0.86 () {56 vars in 1126 bytes} [Mon Aug 29 18:49:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:27.586781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31378f-ac27-443a-9775-2744a939c685 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:27.587130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31378f-ac27-443a-9775-2744a939c685 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:27.587403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31378f-ac27-443a-9775-2744a939c685 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.588952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 475/946] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:27.591863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31378f-ac27-443a-9775-2744a939c685 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:27.622457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:27.643429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:27.665551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4550ee-e44a-46fe-bc62-552622e7cd32 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:27.708299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31378f-ac27-443a-9775-2744a939c685 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:27.708862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf8c84-340c-4cba-8640-ddbe6c50dc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:27.709489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf8c84-340c-4cba-8640-ddbe6c50dc50 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:27.710014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf8c84-340c-4cba-8640-ddbe6c50dc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:27.710285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf8c84-340c-4cba-8640-ddbe6c50dc50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.717877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf8c84-340c-4cba-8640-ddbe6c50dc50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:27.719400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf8c84-340c-4cba-8640-ddbe6c50dc50 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a6bf8c84-340c-4cba-8640-ddbe6c50dc50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:27.731160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:27.732044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:27.732972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:27.732972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:27.732972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:27.732972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:27.732972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:27.732972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:27.732972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:27.732972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:27.732972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 472/947] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:27.734088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94002e18-b3ce-470b-ba84-7f95911ee821 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:27.734088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94002e18-b3ce-470b-ba84-7f95911ee821 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:27.734088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94002e18-b3ce-470b-ba84-7f95911ee821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.750362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafe419-8082-4166-abea-4ec326ae05a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:27.750972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafe419-8082-4166-abea-4ec326ae05a1 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:27.751274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafe419-8082-4166-abea-4ec326ae05a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:27.751540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafe419-8082-4166-abea-4ec326ae05a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.752199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafe419-8082-4166-abea-4ec326ae05a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:27.753549 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafe419-8082-4166-abea-4ec326ae05a1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:27.754008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-94002e18-b3ce-470b-ba84-7f95911ee821 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:27.755295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-94002e18-b3ce-470b-ba84-7f95911ee821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:27.785125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 476/948] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 18:49:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:27.785950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafe419-8082-4166-abea-4ec326ae05a1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:27.864142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 473/949] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:27.938821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:27.942373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafe419-8082-4166-abea-4ec326ae05a1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:27.979534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e5ecd-b10c-4c4d-80f2-ed3fcd018538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:27.980076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e5ecd-b10c-4c4d-80f2-ed3fcd018538 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:27.980347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e5ecd-b10c-4c4d-80f2-ed3fcd018538 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:27.981130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e5ecd-b10c-4c4d-80f2-ed3fcd018538 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:27.981816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e5ecd-b10c-4c4d-80f2-ed3fcd018538 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:27.982114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e5ecd-b10c-4c4d-80f2-ed3fcd018538 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:27.983715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e5ecd-b10c-4c4d-80f2-ed3fcd018538 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:27.998479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 477/950] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:28.002879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94002e18-b3ce-470b-ba84-7f95911ee821 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:28.078769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:28.109852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94002e18-b3ce-470b-ba84-7f95911ee821 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:28.150843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfdbac-8989-41d7-a599-046dc55f9e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:28.151531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfdbac-8989-41d7-a599-046dc55f9e74 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:28.151812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfdbac-8989-41d7-a599-046dc55f9e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:28.152085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfdbac-8989-41d7-a599-046dc55f9e74 None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/7634f750a07143a8b8d91cf0c8e78e16/roles/4ee6c8fb1f294eb0824448e1f26984d6` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:28.154340 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfdbac-8989-41d7-a599-046dc55f9e74 None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=7634f750a07143a8b8d91cf0c8e78e16, role_id=4ee6c8fb1f294eb0824448e1f26984d6)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:28.155798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfdbac-8989-41d7-a599-046dc55f9e74 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:28.162184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e5ecd-b10c-4c4d-80f2-ed3fcd018538 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:28.172668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 474/951] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:49:28 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/7634f750a07143a8b8d91cf0c8e78e16/roles/4ee6c8fb1f294eb0824448e1f26984d6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:49:28.278917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e5ecd-b10c-4c4d-80f2-ed3fcd018538 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:28.318018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887cfe9-2cba-4105-9708-7ec64e3e5ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:28.318788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887cfe9-2cba-4105-9708-7ec64e3e5ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:28.319403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887cfe9-2cba-4105-9708-7ec64e3e5ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:28.319736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887cfe9-2cba-4105-9708-7ec64e3e5ed1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:28.320383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887cfe9-2cba-4105-9708-7ec64e3e5ed1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:28.321800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887cfe9-2cba-4105-9708-7ec64e3e5ed1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8887cfe9-2cba-4105-9708-7ec64e3e5ed1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:28.331024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:28.332319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:28.333441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:28.335774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 478/952] 10.222.0.86 () {56 vars in 1126 bytes} [Mon Aug 29 18:49:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:28.341506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfdbac-8989-41d7-a599-046dc55f9e74 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:28.457409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfdbac-8989-41d7-a599-046dc55f9e74 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:28.469381 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:28.494366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef23ad-dc6a-4290-afdb-c6bf3d368da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:28.494986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef23ad-dc6a-4290-afdb-c6bf3d368da6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:28.495293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef23ad-dc6a-4290-afdb-c6bf3d368da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:28.495571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef23ad-dc6a-4290-afdb-c6bf3d368da6 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:28.496737 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef23ad-dc6a-4290-afdb-c6bf3d368da6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:28.497045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef23ad-dc6a-4290-afdb-c6bf3d368da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:28.498780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef23ad-dc6a-4290-afdb-c6bf3d368da6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:28.503916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 475/953] 10.222.0.86 () {56 vars in 1122 bytes} [Mon Aug 29 18:49:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:28.516363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887cfe9-2cba-4105-9708-7ec64e3e5ed1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:28.542328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:28.627360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887cfe9-2cba-4105-9708-7ec64e3e5ed1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:28.661802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b86d7d-231c-43e3-a50e-5136cbe5d24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:28.662753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b86d7d-231c-43e3-a50e-5136cbe5d24a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:28.663285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b86d7d-231c-43e3-a50e-5136cbe5d24a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:28.663761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b86d7d-231c-43e3-a50e-5136cbe5d24a None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:28.664576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b86d7d-231c-43e3-a50e-5136cbe5d24a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:28.665065 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b86d7d-231c-43e3-a50e-5136cbe5d24a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:28.666622 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b86d7d-231c-43e3-a50e-5136cbe5d24a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:28.677117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 479/954] 10.222.0.86 () {56 vars in 1122 bytes} [Mon Aug 29 18:49:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:28.679068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef23ad-dc6a-4290-afdb-c6bf3d368da6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:28.792121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef23ad-dc6a-4290-afdb-c6bf3d368da6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:28.828082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500abea-1378-4387-8af4-fb1613ca5283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:28.828615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500abea-1378-4387-8af4-fb1613ca5283 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:28.828894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500abea-1378-4387-8af4-fb1613ca5283 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:28.829169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500abea-1378-4387-8af4-fb1613ca5283 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:28.829844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500abea-1378-4387-8af4-fb1613ca5283 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:28.831222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500abea-1378-4387-8af4-fb1613ca5283 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:28.855224 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 476/955] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 18:49:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:29.472845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d021b-0419-459d-ab64-50179c2add89 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:29.473227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d021b-0419-459d-ab64-50179c2add89 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:29.473406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d021b-0419-459d-ab64-50179c2add89 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:29.475033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 480/956] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:29.480362 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a339d-32b1-4b78-9879-a6881764505d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:29.480930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a339d-32b1-4b78-9879-a6881764505d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:29.480930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a339d-32b1-4b78-9879-a6881764505d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:29.499333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-683a339d-32b1-4b78-9879-a6881764505d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:29.500572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-683a339d-32b1-4b78-9879-a6881764505d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:29.524905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:29.580910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 477/957] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:29.852579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d021b-0419-459d-ab64-50179c2add89 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:29.911343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404ce09-dedd-4bfd-a2ec-f735857d6e96 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:29.912003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404ce09-dedd-4bfd-a2ec-f735857d6e96 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:29.912489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404ce09-dedd-4bfd-a2ec-f735857d6e96 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:29.914250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 478/958] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:29.919787 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a18c-b6bb-42de-b63c-8f49f568ab72 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:29.920298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a18c-b6bb-42de-b63c-8f49f568ab72 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:29.920937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a18c-b6bb-42de-b63c-8f49f568ab72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:29.939104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8c61a18c-b6bb-42de-b63c-8f49f568ab72 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:29.940616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8c61a18c-b6bb-42de-b63c-8f49f568ab72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:29.946533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:29.964665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d021b-0419-459d-ab64-50179c2add89 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:30.001245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291d73a-8da2-441b-a5e9-dc33a8e80551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:30.001854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291d73a-8da2-441b-a5e9-dc33a8e80551 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.002129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291d73a-8da2-441b-a5e9-dc33a8e80551 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.002395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291d73a-8da2-441b-a5e9-dc33a8e80551 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.003056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291d73a-8da2-441b-a5e9-dc33a8e80551 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:30.004540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291d73a-8da2-441b-a5e9-dc33a8e80551 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:30.027193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.sql.core [None req-d291d73a-8da2-441b-a5e9-dc33a8e80551 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 18:49:30.027193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 18:49:30.027193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [parameters: {'user_id': 'cd2faa70d5ee4219a840885e0526cc70', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 18:49:30.027193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d291d73a-8da2-441b-a5e9-dc33a8e80551 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Aug 29 18:49:30.028239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:49:30.029141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application yield resource Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise self.value Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application session.commit() Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 18:49:30.030184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.flush() Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application compat.raise_( Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 18:49:30.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:49:30.033857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:49:30.035365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cd2faa70d5ee4219a840885e0526cc70', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:30.036529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 29 18:49:30.037547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 29 18:49:30.044587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 29 18:49:30.044587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 29 18:49:30.044587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 18:49:30.044587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 18:49:30.044587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 18:49:30.044587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:30.044587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 481/960] 10.222.0.86 () {60 vars in 1131 bytes} [Mon Aug 29 18:49:29 2022] POST /identity/v3/users => generated 160 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:30.044587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291d73a-8da2-441b-a5e9-dc33a8e80551 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:30.045681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 479/960] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:30.143013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291d73a-8da2-441b-a5e9-dc33a8e80551 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:30.177091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab515d8-7bae-4299-b560-bbcd2301de5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:30.177715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab515d8-7bae-4299-b560-bbcd2301de5e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.177989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab515d8-7bae-4299-b560-bbcd2301de5e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.178312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab515d8-7bae-4299-b560-bbcd2301de5e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.186554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab515d8-7bae-4299-b560-bbcd2301de5e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:30.188206 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab515d8-7bae-4299-b560-bbcd2301de5e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6ab515d8-7bae-4299-b560-bbcd2301de5e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:30.200425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:30.202642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:30.204910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:30.204910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:30.204910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:30.204910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:30.204910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:30.204910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:30.204910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:30.204910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:30.204910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:49:30.204910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:30.206178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 482/961] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:49:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:30.208565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61a18c-b6bb-42de-b63c-8f49f568ab72 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:30.230519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633cc5a-33b7-4a8d-8c01-dbc71545cba2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.230828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633cc5a-33b7-4a8d-8c01-dbc71545cba2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.231083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633cc5a-33b7-4a8d-8c01-dbc71545cba2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.235585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 483/962] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:30.243576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb590938-03e0-4fa1-ac58-150621595c88 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.243864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb590938-03e0-4fa1-ac58-150621595c88 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.244144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb590938-03e0-4fa1-ac58-150621595c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.262916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bb590938-03e0-4fa1-ac58-150621595c88 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:30.264471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bb590938-03e0-4fa1-ac58-150621595c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:30.275687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:30.312167 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:30.332538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61a18c-b6bb-42de-b63c-8f49f568ab72 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:30.337315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 484/963] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:30.342308 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb590938-03e0-4fa1-ac58-150621595c88 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:30.368642 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254968d6-2eec-4214-863d-0b8e963fdd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:30.369249 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254968d6-2eec-4214-863d-0b8e963fdd22 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.369526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254968d6-2eec-4214-863d-0b8e963fdd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.369818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254968d6-2eec-4214-863d-0b8e963fdd22 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.370488 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254968d6-2eec-4214-863d-0b8e963fdd22 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:30.370842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254968d6-2eec-4214-863d-0b8e963fdd22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:30.372336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254968d6-2eec-4214-863d-0b8e963fdd22 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:30.415240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 480/964] 10.222.0.86 () {56 vars in 1101 bytes} [Mon Aug 29 18:49:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:30.459582 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb590938-03e0-4fa1-ac58-150621595c88 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:30.502282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efaebd-c12e-48ba-aefc-736beae03652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:30.503358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efaebd-c12e-48ba-aefc-736beae03652 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.503861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efaebd-c12e-48ba-aefc-736beae03652 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.504340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efaebd-c12e-48ba-aefc-736beae03652 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.505127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efaebd-c12e-48ba-aefc-736beae03652 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:30.506799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efaebd-c12e-48ba-aefc-736beae03652 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b7efaebd-c12e-48ba-aefc-736beae03652 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:30.517555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:30.518704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:49:30.519741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:30.522341 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 485/965] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:30.528707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254968d6-2eec-4214-863d-0b8e963fdd22 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:30.605860 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efaebd-c12e-48ba-aefc-736beae03652 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:30.608832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:30.647180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254968d6-2eec-4214-863d-0b8e963fdd22 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:30.693240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a03b39-4595-4e12-a9af-c0c8c1c3a59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:30.693869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a03b39-4595-4e12-a9af-c0c8c1c3a59c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.694399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a03b39-4595-4e12-a9af-c0c8c1c3a59c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.694670 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a03b39-4595-4e12-a9af-c0c8c1c3a59c None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.695539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a03b39-4595-4e12-a9af-c0c8c1c3a59c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:30.695664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a03b39-4595-4e12-a9af-c0c8c1c3a59c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:30.696993 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a03b39-4595-4e12-a9af-c0c8c1c3a59c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:30.707211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 481/966] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:30.712936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a03b39-4595-4e12-a9af-c0c8c1c3a59c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:30.726745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:30.751482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efaebd-c12e-48ba-aefc-736beae03652 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:30.786233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9da320-d2f1-4ef0-9f7e-33302402abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:30.786895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9da320-d2f1-4ef0-9f7e-33302402abd5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.787167 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9da320-d2f1-4ef0-9f7e-33302402abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.787437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9da320-d2f1-4ef0-9f7e-33302402abd5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.798989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9da320-d2f1-4ef0-9f7e-33302402abd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4d9da320-d2f1-4ef0-9f7e-33302402abd5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:30.804369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:30.805415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:30.806829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 486/967] 10.222.0.86 () {56 vars in 1120 bytes} [Mon Aug 29 18:49:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:30.811242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9da320-d2f1-4ef0-9f7e-33302402abd5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:30.825076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a03b39-4595-4e12-a9af-c0c8c1c3a59c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:30.859588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f0e87-43a2-4a2a-8a3f-b61def7f22d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:30.860191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f0e87-43a2-4a2a-8a3f-b61def7f22d6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.860472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f0e87-43a2-4a2a-8a3f-b61def7f22d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.860746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f0e87-43a2-4a2a-8a3f-b61def7f22d6 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.861382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f0e87-43a2-4a2a-8a3f-b61def7f22d6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:30.861673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f0e87-43a2-4a2a-8a3f-b61def7f22d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:30.863040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f0e87-43a2-4a2a-8a3f-b61def7f22d6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:30.868060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 482/968] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:30.869062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20555d66-bc4c-45f4-b40f-0f17246b4927 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.869345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20555d66-bc4c-45f4-b40f-0f17246b4927 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.869620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20555d66-bc4c-45f4-b40f-0f17246b4927 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.870834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 483/969] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:30.876079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:30.877255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0140c61-0bb7-4c09-b4b2-01aca3fe72ef None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.877545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0140c61-0bb7-4c09-b4b2-01aca3fe72ef None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.877823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0140c61-0bb7-4c09-b4b2-01aca3fe72ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.898463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e0140c61-0bb7-4c09-b4b2-01aca3fe72ef None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:30.899770 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e0140c61-0bb7-4c09-b4b2-01aca3fe72ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:30.923382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9da320-d2f1-4ef0-9f7e-33302402abd5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:30.962116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203eee78-5aaf-44d8-8d96-5a317c84c6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:30.962814 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eee78-5aaf-44d8-8d96-5a317c84c6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:30.963121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eee78-5aaf-44d8-8d96-5a317c84c6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:30.963411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eee78-5aaf-44d8-8d96-5a317c84c6d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:30.964058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203eee78-5aaf-44d8-8d96-5a317c84c6d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:30.964358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203eee78-5aaf-44d8-8d96-5a317c84c6d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:30.969865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203eee78-5aaf-44d8-8d96-5a317c84c6d6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:30.990276 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 484/970] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:30.990518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 487/971] 10.222.0.86 () {56 vars in 1115 bytes} [Mon Aug 29 18:49:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:30.991040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0140c61-0bb7-4c09-b4b2-01aca3fe72ef None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:30.991040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:31.098627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0140c61-0bb7-4c09-b4b2-01aca3fe72ef None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:31.134137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef2dac-5cef-4623-a6ec-9dbf5dc1e919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:31.134796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef2dac-5cef-4623-a6ec-9dbf5dc1e919 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:31.135113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef2dac-5cef-4623-a6ec-9dbf5dc1e919 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:31.135393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef2dac-5cef-4623-a6ec-9dbf5dc1e919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:31.145362 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef2dac-5cef-4623-a6ec-9dbf5dc1e919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:31.146808 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef2dac-5cef-4623-a6ec-9dbf5dc1e919 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-edef2dac-5cef-4623-a6ec-9dbf5dc1e919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:31.161182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:31.162156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:31.168471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:31.168471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:31.168471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:31.168471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 485/972] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:31.170252 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203eee78-5aaf-44d8-8d96-5a317c84c6d6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:31.257217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef2dac-5cef-4623-a6ec-9dbf5dc1e919 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:31.282543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203eee78-5aaf-44d8-8d96-5a317c84c6d6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:31.310306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:31.326153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcbe08-a626-4883-aa9e-45cb10d17cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:31.326767 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcbe08-a626-4883-aa9e-45cb10d17cde None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:31.327073 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcbe08-a626-4883-aa9e-45cb10d17cde None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:31.327349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcbe08-a626-4883-aa9e-45cb10d17cde None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:31.327628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:31.327994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcbe08-a626-4883-aa9e-45cb10d17cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:31.328533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcbe08-a626-4883-aa9e-45cb10d17cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:31.329733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcbe08-a626-4883-aa9e-45cb10d17cde None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:31.348740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 488/973] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:31.354022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcbe08-a626-4883-aa9e-45cb10d17cde None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:31.383649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef2dac-5cef-4623-a6ec-9dbf5dc1e919 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:31.421131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d734bc-3097-463e-bff7-0b748c3fba3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:31.421806 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d734bc-3097-463e-bff7-0b748c3fba3e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:31.422088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d734bc-3097-463e-bff7-0b748c3fba3e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:31.422319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d734bc-3097-463e-bff7-0b748c3fba3e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:31.422926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d734bc-3097-463e-bff7-0b748c3fba3e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:31.424277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d734bc-3097-463e-bff7-0b748c3fba3e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-29d734bc-3097-463e-bff7-0b748c3fba3e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:31.433785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:31.434980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:31.436021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 486/974] 10.222.0.86 () {56 vars in 1126 bytes} [Mon Aug 29 18:49:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:31.440485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d734bc-3097-463e-bff7-0b748c3fba3e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:31.461634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcbe08-a626-4883-aa9e-45cb10d17cde None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:31.500779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d335b-441f-4db7-b33c-5d026bf7274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:31.501514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d335b-441f-4db7-b33c-5d026bf7274d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:31.501857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d335b-441f-4db7-b33c-5d026bf7274d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:31.502202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d335b-441f-4db7-b33c-5d026bf7274d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:31.510068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d335b-441f-4db7-b33c-5d026bf7274d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:31.511481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d335b-441f-4db7-b33c-5d026bf7274d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d49d335b-441f-4db7-b33c-5d026bf7274d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:31.523181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:31.524183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:31.526026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:31.526026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:31.526026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:31.526026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:31.526026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:31.526026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:31.526026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:31.526026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:31.526026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:49:31.526026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:31.526026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 489/975] 10.222.0.86 () {56 vars in 1117 bytes} [Mon Aug 29 18:49:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:31.530453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d335b-441f-4db7-b33c-5d026bf7274d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:31.550563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d734bc-3097-463e-bff7-0b748c3fba3e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:31.568828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:31.579208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:31.588868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d928b2-0904-4b9c-b0ca-539c8c924f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:31.589476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d928b2-0904-4b9c-b0ca-539c8c924f53 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:31.589782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d928b2-0904-4b9c-b0ca-539c8c924f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:31.590567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d928b2-0904-4b9c-b0ca-539c8c924f53 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:31.591411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d928b2-0904-4b9c-b0ca-539c8c924f53 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:31.591828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d928b2-0904-4b9c-b0ca-539c8c924f53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:31.593240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d928b2-0904-4b9c-b0ca-539c8c924f53 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:31.599288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 487/976] 10.222.0.86 () {56 vars in 1122 bytes} [Mon Aug 29 18:49:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:31.604410 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d928b2-0904-4b9c-b0ca-539c8c924f53 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:31.679881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d335b-441f-4db7-b33c-5d026bf7274d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:31.720805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef8555-1d11-4fd8-9f40-f29be0cb0fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:31.721419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef8555-1d11-4fd8-9f40-f29be0cb0fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:31.721697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef8555-1d11-4fd8-9f40-f29be0cb0fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:31.721806 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d928b2-0904-4b9c-b0ca-539c8c924f53 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:31.721966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef8555-1d11-4fd8-9f40-f29be0cb0fc6 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:31.722688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef8555-1d11-4fd8-9f40-f29be0cb0fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:31.723042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef8555-1d11-4fd8-9f40-f29be0cb0fc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:31.724666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef8555-1d11-4fd8-9f40-f29be0cb0fc6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:31.743200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 490/977] 10.222.0.86 () {56 vars in 1149 bytes} [Mon Aug 29 18:49:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:31.748562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef8555-1d11-4fd8-9f40-f29be0cb0fc6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:31.760577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571d8c3-ed13-44ea-8c30-0e09e156dfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:31.761482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d8c3-ed13-44ea-8c30-0e09e156dfce None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:31.762570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d8c3-ed13-44ea-8c30-0e09e156dfce None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:31.763163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d8c3-ed13-44ea-8c30-0e09e156dfce None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:31.764157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571d8c3-ed13-44ea-8c30-0e09e156dfce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:31.764468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571d8c3-ed13-44ea-8c30-0e09e156dfce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:31.766061 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571d8c3-ed13-44ea-8c30-0e09e156dfce None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:31.776347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 488/978] 10.222.0.86 () {56 vars in 1122 bytes} [Mon Aug 29 18:49:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:31.776988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571d8c3-ed13-44ea-8c30-0e09e156dfce None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:31.826830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:31.850863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef8555-1d11-4fd8-9f40-f29be0cb0fc6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:31.852443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:31.884840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571d8c3-ed13-44ea-8c30-0e09e156dfce None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:31.887763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162256b6-1e44-45c0-bd16-46d70ad96a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:31.888378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162256b6-1e44-45c0-bd16-46d70ad96a44 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:31.888664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162256b6-1e44-45c0-bd16-46d70ad96a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:31.888933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162256b6-1e44-45c0-bd16-46d70ad96a44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:31.922249 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35408d51-5f27-4b31-928d-7665ede93fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:31.923511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35408d51-5f27-4b31-928d-7665ede93fdc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:31.923798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35408d51-5f27-4b31-928d-7665ede93fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:31.924072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35408d51-5f27-4b31-928d-7665ede93fdc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:31.924781 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35408d51-5f27-4b31-928d-7665ede93fdc None admin] RBAC: Adding query filter params (interface=admin, service_id=f70e89ed0a9648b0a8ba8dcc3124d903, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:31.925093 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35408d51-5f27-4b31-928d-7665ede93fdc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:31.926455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35408d51-5f27-4b31-928d-7665ede93fdc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:31.932270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 489/979] 10.222.0.86 () {56 vars in 1249 bytes} [Mon Aug 29 18:49:31 2022] GET /identity/v3/endpoints?service_id=f70e89ed0a9648b0a8ba8dcc3124d903&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:31.937345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35408d51-5f27-4b31-928d-7665ede93fdc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:31.964095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162256b6-1e44-45c0-bd16-46d70ad96a44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:31.966096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162256b6-1e44-45c0-bd16-46d70ad96a44 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:32.045452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35408d51-5f27-4b31-928d-7665ede93fdc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:32.056472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:32.074210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 491/980] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:31 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:49:32.075862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162256b6-1e44-45c0-bd16-46d70ad96a44 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:32.077778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:32.088903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac35cb4-c8d8-4aba-815d-c89dbf6ed1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:32.089511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac35cb4-c8d8-4aba-815d-c89dbf6ed1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:32.089812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac35cb4-c8d8-4aba-815d-c89dbf6ed1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:32.090082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac35cb4-c8d8-4aba-815d-c89dbf6ed1f3 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:32.090671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac35cb4-c8d8-4aba-815d-c89dbf6ed1f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:32.092061 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac35cb4-c8d8-4aba-815d-c89dbf6ed1f3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:32.098136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 490/981] 10.222.0.86 () {56 vars in 1087 bytes} [Mon Aug 29 18:49:31 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 18:49:32.182857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162256b6-1e44-45c0-bd16-46d70ad96a44 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:32.220638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68109c-6faf-47da-966e-ac8eabf3eb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:32.221291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68109c-6faf-47da-966e-ac8eabf3eb9a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:32.221571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68109c-6faf-47da-966e-ac8eabf3eb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:32.221843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68109c-6faf-47da-966e-ac8eabf3eb9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:32.232998 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68109c-6faf-47da-966e-ac8eabf3eb9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:32.234499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68109c-6faf-47da-966e-ac8eabf3eb9a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6f68109c-6faf-47da-966e-ac8eabf3eb9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:32.250309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:32.251472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:32.252398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:32.252398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:32.252398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:32.252398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 492/982] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:32.257333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac35cb4-c8d8-4aba-815d-c89dbf6ed1f3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:32.363733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac35cb4-c8d8-4aba-815d-c89dbf6ed1f3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:32.397654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c54b7-3566-4f32-a4f0-3336aed82e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:32.398416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c54b7-3566-4f32-a4f0-3336aed82e60 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:32.398797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c54b7-3566-4f32-a4f0-3336aed82e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:32.399201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c54b7-3566-4f32-a4f0-3336aed82e60 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:32.399944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c54b7-3566-4f32-a4f0-3336aed82e60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:32.400350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c54b7-3566-4f32-a4f0-3336aed82e60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:32.401800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c54b7-3566-4f32-a4f0-3336aed82e60 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:32.414146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 491/983] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:32.420117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68109c-6faf-47da-966e-ac8eabf3eb9a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:32.478933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:32.527079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68109c-6faf-47da-966e-ac8eabf3eb9a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:32.562123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11a9ad-662b-4e8a-b8a7-e1fccd60ab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:32.563169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11a9ad-662b-4e8a-b8a7-e1fccd60ab98 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:32.563658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11a9ad-662b-4e8a-b8a7-e1fccd60ab98 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:32.564132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11a9ad-662b-4e8a-b8a7-e1fccd60ab98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:32.571558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11a9ad-662b-4e8a-b8a7-e1fccd60ab98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:32.573300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11a9ad-662b-4e8a-b8a7-e1fccd60ab98 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1b11a9ad-662b-4e8a-b8a7-e1fccd60ab98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:32.585057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:32.585986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:32.586908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:32.586908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:32.586908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:32.586908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:32.586908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:32.586908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:32.586908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:32.586908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:32.593293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 493/984] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:32.594658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c54b7-3566-4f32-a4f0-3336aed82e60 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:32.631170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:32.706901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c54b7-3566-4f32-a4f0-3336aed82e60 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:32.743821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e052c-ffd7-45d2-bc1a-cfece24a588e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:32.744436 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e052c-ffd7-45d2-bc1a-cfece24a588e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:32.744711 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e052c-ffd7-45d2-bc1a-cfece24a588e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:32.744985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e052c-ffd7-45d2-bc1a-cfece24a588e None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:32.745682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e052c-ffd7-45d2-bc1a-cfece24a588e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:32.746007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e052c-ffd7-45d2-bc1a-cfece24a588e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:32.747487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e052c-ffd7-45d2-bc1a-cfece24a588e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:32.764618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 492/985] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:32.766020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11a9ad-662b-4e8a-b8a7-e1fccd60ab98 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:32.818934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106195cb-f5f6-4569-b9e1-fd97d00d6d1b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:32.819500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106195cb-f5f6-4569-b9e1-fd97d00d6d1b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:32.819781 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106195cb-f5f6-4569-b9e1-fd97d00d6d1b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:32.821239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 493/986] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:32.825821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2be442-c617-48d2-ad94-d997bb18bc4e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:32.826110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2be442-c617-48d2-ad94-d997bb18bc4e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:32.826387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2be442-c617-48d2-ad94-d997bb18bc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:32.844598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cc2be442-c617-48d2-ad94-d997bb18bc4e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:32.845794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cc2be442-c617-48d2-ad94-d997bb18bc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:32.871873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11a9ad-662b-4e8a-b8a7-e1fccd60ab98 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:32.885307 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:32.907334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08e6f4-fd02-4d09-9a34-c2f30eeb0ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:32.907926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08e6f4-fd02-4d09-9a34-c2f30eeb0ead None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:32.908194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08e6f4-fd02-4d09-9a34-c2f30eeb0ead None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:32.908462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08e6f4-fd02-4d09-9a34-c2f30eeb0ead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:32.909169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08e6f4-fd02-4d09-9a34-c2f30eeb0ead None admin] RBAC: Adding query filter params (role.id=4ee6c8fb1f294eb0824448e1f26984d6, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=7634f750a07143a8b8d91cf0c8e78e16) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:32.909469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08e6f4-fd02-4d09-9a34-c2f30eeb0ead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:32.910877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08e6f4-fd02-4d09-9a34-c2f30eeb0ead None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:32.913576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 494/987] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:32.927295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 494/988] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:32 2022] GET /identity/v3/role_assignments?user.id=7634f750a07143a8b8d91cf0c8e78e16&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=4ee6c8fb1f294eb0824448e1f26984d6 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:33.185712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2be442-c617-48d2-ad94-d997bb18bc4e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:33.213144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:33.291921 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2be442-c617-48d2-ad94-d997bb18bc4e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:33.326033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41566966-a9b5-4784-b790-d5e110fa5b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:33.326638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41566966-a9b5-4784-b790-d5e110fa5b18 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:33.326919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41566966-a9b5-4784-b790-d5e110fa5b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:33.327222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41566966-a9b5-4784-b790-d5e110fa5b18 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:33.327879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41566966-a9b5-4784-b790-d5e110fa5b18 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:33.329232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41566966-a9b5-4784-b790-d5e110fa5b18 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:33.345287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 495/989] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:49:33 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:34.034908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa049a-4249-4a87-9399-2332ed35d421 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:34.035479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa049a-4249-4a87-9399-2332ed35d421 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:34.035479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa049a-4249-4a87-9399-2332ed35d421 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:34.036803 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 495/990] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:34.042115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e552-4ad6-4bf5-aafa-f64124f8687a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:34.042506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e552-4ad6-4bf5-aafa-f64124f8687a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:34.042676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e552-4ad6-4bf5-aafa-f64124f8687a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:34.061625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e649e552-4ad6-4bf5-aafa-f64124f8687a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:34.062801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e649e552-4ad6-4bf5-aafa-f64124f8687a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:34.136576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 496/991] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:34.405384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa049a-4249-4a87-9399-2332ed35d421 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:34.510227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa049a-4249-4a87-9399-2332ed35d421 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:34.544418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32b62d-781d-4613-90ed-ada4101a289e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:34.545043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b62d-781d-4613-90ed-ada4101a289e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:34.545367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b62d-781d-4613-90ed-ada4101a289e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:34.545808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b62d-781d-4613-90ed-ada4101a289e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:34.546379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32b62d-781d-4613-90ed-ada4101a289e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:34.547760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32b62d-781d-4613-90ed-ada4101a289e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7a32b62d-781d-4613-90ed-ada4101a289e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:34.556422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:34.557426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:49:34.558274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:34.562418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 496/992] 10.222.0.86 () {56 vars in 1126 bytes} [Mon Aug 29 18:49:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:34.563776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649e552-4ad6-4bf5-aafa-f64124f8687a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:34.576267 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:34.668416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649e552-4ad6-4bf5-aafa-f64124f8687a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:34.711608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f755a-b161-480c-bd2a-6dd8610e3610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:34.712222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f755a-b161-480c-bd2a-6dd8610e3610 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:34.712498 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f755a-b161-480c-bd2a-6dd8610e3610 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:34.712768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f755a-b161-480c-bd2a-6dd8610e3610 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:34.713390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f755a-b161-480c-bd2a-6dd8610e3610 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:34.713688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f755a-b161-480c-bd2a-6dd8610e3610 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:34.715692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f755a-b161-480c-bd2a-6dd8610e3610 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:34.720837 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 497/993] 10.222.0.86 () {56 vars in 1122 bytes} [Mon Aug 29 18:49:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:34.726424 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32b62d-781d-4613-90ed-ada4101a289e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:34.772602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:34.832737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32b62d-781d-4613-90ed-ada4101a289e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:34.857657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caca74-54cf-4527-96f6-3a4ee0579250 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:34.858185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caca74-54cf-4527-96f6-3a4ee0579250 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:34.858465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caca74-54cf-4527-96f6-3a4ee0579250 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:34.859972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 498/994] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:34.864755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56496ba1-14e3-457a-a7b6-0a34f8d06868 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:34.865046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56496ba1-14e3-457a-a7b6-0a34f8d06868 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:34.865325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56496ba1-14e3-457a-a7b6-0a34f8d06868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:34.870841 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f564f1-0964-4663-a20b-fb934e1a3c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:34.872298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f564f1-0964-4663-a20b-fb934e1a3c10 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:34.872751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f564f1-0964-4663-a20b-fb934e1a3c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:34.873207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f564f1-0964-4663-a20b-fb934e1a3c10 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:34.874074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f564f1-0964-4663-a20b-fb934e1a3c10 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:34.874546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f564f1-0964-4663-a20b-fb934e1a3c10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:34.876120 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f564f1-0964-4663-a20b-fb934e1a3c10 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:34.885977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 497/995] 10.222.0.86 () {56 vars in 1122 bytes} [Mon Aug 29 18:49:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:34.886151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-56496ba1-14e3-457a-a7b6-0a34f8d06868 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:34.886151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-56496ba1-14e3-457a-a7b6-0a34f8d06868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:34.888037 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f564f1-0964-4663-a20b-fb934e1a3c10 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:34.956371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 499/996] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:34.995767 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f564f1-0964-4663-a20b-fb934e1a3c10 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:35.003811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:35.032062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cd400-ea05-4c99-a0eb-84b27fcfe976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:35.032664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cd400-ea05-4c99-a0eb-84b27fcfe976 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:35.032937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cd400-ea05-4c99-a0eb-84b27fcfe976 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:35.033200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cd400-ea05-4c99-a0eb-84b27fcfe976 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:35.033849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cd400-ea05-4c99-a0eb-84b27fcfe976 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:35.035222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cd400-ea05-4c99-a0eb-84b27fcfe976 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:35.053102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 498/997] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 18:49:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:49:35.226312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56496ba1-14e3-457a-a7b6-0a34f8d06868 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:35.229402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:35.308584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525c85b-0cd5-42a1-9c0e-3b5177b0b4bf None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:35.308941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525c85b-0cd5-42a1-9c0e-3b5177b0b4bf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:35.309139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525c85b-0cd5-42a1-9c0e-3b5177b0b4bf None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:35.310753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 499/998] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:35.316131 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a37dc-3fb6-4e8d-a5ff-ad20f6d0d17f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:35.316420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a37dc-3fb6-4e8d-a5ff-ad20f6d0d17f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:35.316698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a37dc-3fb6-4e8d-a5ff-ad20f6d0d17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:35.334506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56496ba1-14e3-457a-a7b6-0a34f8d06868 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:35.335315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-889a37dc-3fb6-4e8d-a5ff-ad20f6d0d17f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:35.336512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-889a37dc-3fb6-4e8d-a5ff-ad20f6d0d17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:35.379837 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056e209-888a-4062-a76d-26b54aaebd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:35.380324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056e209-888a-4062-a76d-26b54aaebd6c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:35.380606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056e209-888a-4062-a76d-26b54aaebd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:35.380877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056e209-888a-4062-a76d-26b54aaebd6c None admin] PATH_INFO: `/v3/services/image` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:35.381454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056e209-888a-4062-a76d-26b54aaebd6c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:35.382798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056e209-888a-4062-a76d-26b54aaebd6c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5056e209-888a-4062-a76d-26b54aaebd6c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:35.394790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:35.395890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:35.396733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 500/999] 10.222.0.86 () {56 vars in 1105 bytes} [Mon Aug 29 18:49:35 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:35.401747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056e209-888a-4062-a76d-26b54aaebd6c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:35.419067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 500/1000] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:35.466893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:35.506895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056e209-888a-4062-a76d-26b54aaebd6c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:35.543094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca61b0-3b17-423f-9228-11e64367f6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:35.543704 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca61b0-3b17-423f-9228-11e64367f6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:35.543983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca61b0-3b17-423f-9228-11e64367f6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:35.544255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca61b0-3b17-423f-9228-11e64367f6b5 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:35.544877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca61b0-3b17-423f-9228-11e64367f6b5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:35.545182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca61b0-3b17-423f-9228-11e64367f6b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:35.546678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca61b0-3b17-423f-9228-11e64367f6b5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:35.556103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 501/1001] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:35.557435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a37dc-3fb6-4e8d-a5ff-ad20f6d0d17f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:35.650662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:35.658698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a37dc-3fb6-4e8d-a5ff-ad20f6d0d17f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:35.688710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca61b0-3b17-423f-9228-11e64367f6b5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:35.697653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556bf33-d67e-406b-83f5-993319ae7809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:35.698271 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556bf33-d67e-406b-83f5-993319ae7809 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:35.698550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556bf33-d67e-406b-83f5-993319ae7809 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:35.698820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556bf33-d67e-406b-83f5-993319ae7809 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:35.700025 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556bf33-d67e-406b-83f5-993319ae7809 None admin] RBAC: Adding query filter params (type=image) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:35.700333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556bf33-d67e-406b-83f5-993319ae7809 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:35.701805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556bf33-d67e-406b-83f5-993319ae7809 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:35.707134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 501/1002] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:35.812392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca61b0-3b17-423f-9228-11e64367f6b5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:35.841325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:35.853747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b259e9-3347-4065-b68c-271107e81e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:35.854119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b259e9-3347-4065-b68c-271107e81e30 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:35.854402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b259e9-3347-4065-b68c-271107e81e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:35.854675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b259e9-3347-4065-b68c-271107e81e30 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:35.855405 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b259e9-3347-4065-b68c-271107e81e30 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:35.856866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b259e9-3347-4065-b68c-271107e81e30 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-36b259e9-3347-4065-b68c-271107e81e30 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:35.865980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:35.866963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:49:35.867882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:35.872689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 502/1003] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:35.873691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556bf33-d67e-406b-83f5-993319ae7809 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:35.974236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556bf33-d67e-406b-83f5-993319ae7809 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:36.008031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031dee6b-11b9-422a-9442-1780e58fe865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:36.008639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dee6b-11b9-422a-9442-1780e58fe865 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:36.008918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dee6b-11b9-422a-9442-1780e58fe865 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:36.009188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dee6b-11b9-422a-9442-1780e58fe865 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:36.009808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031dee6b-11b9-422a-9442-1780e58fe865 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:36.010110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031dee6b-11b9-422a-9442-1780e58fe865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:36.011506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031dee6b-11b9-422a-9442-1780e58fe865 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:36.017059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 502/1004] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:36.022488 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b259e9-3347-4065-b68c-271107e81e30 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:36.126353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b259e9-3347-4065-b68c-271107e81e30 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:36.160068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508904e3-a420-4e5f-ae94-76f3020c3fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:36.160776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508904e3-a420-4e5f-ae94-76f3020c3fac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:36.161057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508904e3-a420-4e5f-ae94-76f3020c3fac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:36.161328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508904e3-a420-4e5f-ae94-76f3020c3fac None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:36.161975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508904e3-a420-4e5f-ae94-76f3020c3fac None admin] RBAC: Adding query filter params (type=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:36.162279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508904e3-a420-4e5f-ae94-76f3020c3fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:36.163707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508904e3-a420-4e5f-ae94-76f3020c3fac None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:36.170076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 503/1005] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:36.175686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031dee6b-11b9-422a-9442-1780e58fe865 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:36.205351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:36.280060 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031dee6b-11b9-422a-9442-1780e58fe865 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:36.315358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494e739-15b3-4a86-8202-ae4999e527cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:36.315966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494e739-15b3-4a86-8202-ae4999e527cd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:36.316244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494e739-15b3-4a86-8202-ae4999e527cd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:36.316516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494e739-15b3-4a86-8202-ae4999e527cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:36.317213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494e739-15b3-4a86-8202-ae4999e527cd None admin] RBAC: Adding query filter params (interface=public, service_id=74c293b8fd4f4d988c924e05bd5bae91, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:36.317519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494e739-15b3-4a86-8202-ae4999e527cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:36.318884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494e739-15b3-4a86-8202-ae4999e527cd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:36.324636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 503/1006] 10.222.0.86 () {56 vars in 1251 bytes} [Mon Aug 29 18:49:36 2022] GET /identity/v3/endpoints?service_id=74c293b8fd4f4d988c924e05bd5bae91&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:36.329895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508904e3-a420-4e5f-ae94-76f3020c3fac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:36.386551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:36.430778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508904e3-a420-4e5f-ae94-76f3020c3fac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:36.464528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebbf31-d854-43f3-8933-5ec2fbdd7347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:36.465715 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebbf31-d854-43f3-8933-5ec2fbdd7347 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:36.465997 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebbf31-d854-43f3-8933-5ec2fbdd7347 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:36.466266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebbf31-d854-43f3-8933-5ec2fbdd7347 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:36.466862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebbf31-d854-43f3-8933-5ec2fbdd7347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:36.468247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebbf31-d854-43f3-8933-5ec2fbdd7347 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:36.474631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 504/1007] 10.222.0.86 () {56 vars in 1087 bytes} [Mon Aug 29 18:49:36 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:49:37.029710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08df18a-7327-450c-a4a1-e357d5ec091a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:37.030029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08df18a-7327-450c-a4a1-e357d5ec091a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:37.030236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08df18a-7327-450c-a4a1-e357d5ec091a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:37.031732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 504/1008] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:37.036608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f57e8d-9dd4-4fcd-ac1f-26777c996ad0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:37.036903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f57e8d-9dd4-4fcd-ac1f-26777c996ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:37.037176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f57e8d-9dd4-4fcd-ac1f-26777c996ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:37.055834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b1f57e8d-9dd4-4fcd-ac1f-26777c996ad0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:37.057016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b1f57e8d-9dd4-4fcd-ac1f-26777c996ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:37.122433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 505/1009] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:37 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:37.389609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08df18a-7327-450c-a4a1-e357d5ec091a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:37.480504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:37.494316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08df18a-7327-450c-a4a1-e357d5ec091a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:37.528894 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde9e8-6af8-4cfe-9225-568e6c654787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:37.529529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde9e8-6af8-4cfe-9225-568e6c654787 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:37.529807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde9e8-6af8-4cfe-9225-568e6c654787 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:37.530082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde9e8-6af8-4cfe-9225-568e6c654787 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:37.531217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffde9e8-6af8-4cfe-9225-568e6c654787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:37.532648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffde9e8-6af8-4cfe-9225-568e6c654787 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:37.538956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 505/1010] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:37.539950 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f57e8d-9dd4-4fcd-ac1f-26777c996ad0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:37.592562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:37.643156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f57e8d-9dd4-4fcd-ac1f-26777c996ad0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:37.671560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10f8a3-636e-4640-9395-3bbbc933504d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:37.671862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10f8a3-636e-4640-9395-3bbbc933504d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:37.672106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10f8a3-636e-4640-9395-3bbbc933504d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:37.673431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 506/1011] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:37.678500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bb2c6-6b35-4277-86ba-063fc6927a59 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:37.678618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbab2f2-71a3-43c0-b6de-175511dc637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:37.678863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bb2c6-6b35-4277-86ba-063fc6927a59 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:37.679098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bb2c6-6b35-4277-86ba-063fc6927a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:37.679197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab2f2-71a3-43c0-b6de-175511dc637f None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:37.682325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab2f2-71a3-43c0-b6de-175511dc637f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:37.682325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab2f2-71a3-43c0-b6de-175511dc637f None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:37.682325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbab2f2-71a3-43c0-b6de-175511dc637f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:37.682325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbab2f2-71a3-43c0-b6de-175511dc637f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:37.703280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bd0bb2c6-6b35-4277-86ba-063fc6927a59 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:37.704867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bd0bb2c6-6b35-4277-86ba-063fc6927a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:37.714553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 506/1012] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:49:37 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:49:37.795048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 507/1013] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:37 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:38.060546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbab2f2-71a3-43c0-b6de-175511dc637f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:38.164131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbab2f2-71a3-43c0-b6de-175511dc637f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:38.166043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:38.202344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879ad5e-453a-4ac8-b5f9-4cee416f22f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:38.202954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879ad5e-453a-4ac8-b5f9-4cee416f22f5 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:38.203257 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879ad5e-453a-4ac8-b5f9-4cee416f22f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:38.203529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879ad5e-453a-4ac8-b5f9-4cee416f22f5 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:38.204172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879ad5e-453a-4ac8-b5f9-4cee416f22f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:38.205527 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879ad5e-453a-4ac8-b5f9-4cee416f22f5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:38.222491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 507/1014] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:49:38 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:38.421696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd6e4e-d4e3-495d-b5d3-623d5a43529c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:38.422193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd6e4e-d4e3-495d-b5d3-623d5a43529c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:38.422193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd6e4e-d4e3-495d-b5d3-623d5a43529c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:38.423519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 508/1015] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:38.428532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137273e-5513-4f32-a209-1476c4fb7d7e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:38.428823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137273e-5513-4f32-a209-1476c4fb7d7e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:38.429099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137273e-5513-4f32-a209-1476c4fb7d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:38.448006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7137273e-5513-4f32-a209-1476c4fb7d7e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:38.449208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7137273e-5513-4f32-a209-1476c4fb7d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:38.521846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 508/1016] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:38 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:38.788916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd6e4e-d4e3-495d-b5d3-623d5a43529c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:38.878935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:38.895815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd6e4e-d4e3-495d-b5d3-623d5a43529c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:38.930828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d78bd-e395-4de9-9979-d6f7874e91ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:38.931469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d78bd-e395-4de9-9979-d6f7874e91ea None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:38.931744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d78bd-e395-4de9-9979-d6f7874e91ea None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:38.932008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d78bd-e395-4de9-9979-d6f7874e91ea None admin] PATH_INFO: `/v3/services/compute` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:38.932575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d78bd-e395-4de9-9979-d6f7874e91ea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:38.933969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d78bd-e395-4de9-9979-d6f7874e91ea None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-472d78bd-e395-4de9-9979-d6f7874e91ea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:38.942534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:38.943713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:49:38.945176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:38.946266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 509/1017] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:38.949693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137273e-5513-4f32-a209-1476c4fb7d7e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:39.035204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:39.051336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137273e-5513-4f32-a209-1476c4fb7d7e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:39.086169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4e3ad-3eca-4264-b6fc-e5334afde5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:39.086968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4e3ad-3eca-4264-b6fc-e5334afde5b4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:39.087297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4e3ad-3eca-4264-b6fc-e5334afde5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:39.087570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4e3ad-3eca-4264-b6fc-e5334afde5b4 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:39.088238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4e3ad-3eca-4264-b6fc-e5334afde5b4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:39.088537 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4e3ad-3eca-4264-b6fc-e5334afde5b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:39.090438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4e3ad-3eca-4264-b6fc-e5334afde5b4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:39.095775 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 509/1018] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:39.101735 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d78bd-e395-4de9-9979-d6f7874e91ea None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:39.203924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d78bd-e395-4de9-9979-d6f7874e91ea None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:39.237459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d7e5f-3873-405f-aafe-4850eee259b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:39.238064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d7e5f-3873-405f-aafe-4850eee259b7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:39.238339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d7e5f-3873-405f-aafe-4850eee259b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:39.238606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d7e5f-3873-405f-aafe-4850eee259b7 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:39.239257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d7e5f-3873-405f-aafe-4850eee259b7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:39.239556 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d7e5f-3873-405f-aafe-4850eee259b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:39.241268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d7e5f-3873-405f-aafe-4850eee259b7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:39.246377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 510/1019] 10.222.0.86 () {56 vars in 1112 bytes} [Mon Aug 29 18:49:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:39.251939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4e3ad-3eca-4264-b6fc-e5334afde5b4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:39.355983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4e3ad-3eca-4264-b6fc-e5334afde5b4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:39.390335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f89fb9-b4f3-4af7-a3c8-da6b06487e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:39.390934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f89fb9-b4f3-4af7-a3c8-da6b06487e21 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:39.391235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f89fb9-b4f3-4af7-a3c8-da6b06487e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:39.391508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f89fb9-b4f3-4af7-a3c8-da6b06487e21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:39.392166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f89fb9-b4f3-4af7-a3c8-da6b06487e21 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:39.393507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f89fb9-b4f3-4af7-a3c8-da6b06487e21 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:39.419036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 510/1020] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 18:49:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:39.676628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7a426-32a3-4b62-9d49-314214f28f42 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:39.676628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7a426-32a3-4b62-9d49-314214f28f42 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:39.676628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7a426-32a3-4b62-9d49-314214f28f42 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:39.676628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 511/1021] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:39.684332 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8acd7c-af14-4307-b1c7-c70031675e29 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:39.684598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8acd7c-af14-4307-b1c7-c70031675e29 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:39.684814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8acd7c-af14-4307-b1c7-c70031675e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:39.703747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1a8acd7c-af14-4307-b1c7-c70031675e29 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:39.704940 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1a8acd7c-af14-4307-b1c7-c70031675e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:39.706958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:39.782830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 511/1022] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:40.054175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7a426-32a3-4b62-9d49-314214f28f42 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:40.071458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:40.138064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886333f-7b99-4708-a97e-efca9e6ec209 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:40.138588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886333f-7b99-4708-a97e-efca9e6ec209 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:40.139066 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886333f-7b99-4708-a97e-efca9e6ec209 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:40.141509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 512/1023] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:40.147201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672f621-ae08-4f9a-94e7-71d14b747cb1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:40.147555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672f621-ae08-4f9a-94e7-71d14b747cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:40.147737 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672f621-ae08-4f9a-94e7-71d14b747cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:40.168269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7a426-32a3-4b62-9d49-314214f28f42 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:40.172260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4672f621-ae08-4f9a-94e7-71d14b747cb1 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:40.174013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4672f621-ae08-4f9a-94e7-71d14b747cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:40.205092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8386c-0aba-4199-b466-c50b3413590e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:40.205728 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8386c-0aba-4199-b466-c50b3413590e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:40.206005 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8386c-0aba-4199-b466-c50b3413590e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:40.206275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8386c-0aba-4199-b466-c50b3413590e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:40.207423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8386c-0aba-4199-b466-c50b3413590e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:40.208840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8386c-0aba-4199-b466-c50b3413590e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:40.210414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 512/1024] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:40.217902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8386c-0aba-4199-b466-c50b3413590e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:40.265455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 513/1025] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:40.303197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:40.323071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8386c-0aba-4199-b466-c50b3413590e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:40.357977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a156b2-85a0-4293-bb95-ee707b57a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:40.358592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a156b2-85a0-4293-bb95-ee707b57a186 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:40.358867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a156b2-85a0-4293-bb95-ee707b57a186 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:40.359167 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a156b2-85a0-4293-bb95-ee707b57a186 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:40.359814 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a156b2-85a0-4293-bb95-ee707b57a186 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:40.361220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a156b2-85a0-4293-bb95-ee707b57a186 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:40.384281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 513/1026] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:49:40 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:49:40.538716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672f621-ae08-4f9a-94e7-71d14b747cb1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:40.616991 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:40.655732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672f621-ae08-4f9a-94e7-71d14b747cb1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:40.695710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431c59e-92fc-4958-9999-444271002fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:40.696344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431c59e-92fc-4958-9999-444271002fbc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:40.696620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431c59e-92fc-4958-9999-444271002fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:40.696885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431c59e-92fc-4958-9999-444271002fbc None admin] PATH_INFO: `/v3/services/image` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:40.697458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431c59e-92fc-4958-9999-444271002fbc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:40.698809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431c59e-92fc-4958-9999-444271002fbc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0431c59e-92fc-4958-9999-444271002fbc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:40.707597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:40.708587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:49:40.709542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:40.710827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 514/1027] 10.222.0.86 () {56 vars in 1105 bytes} [Mon Aug 29 18:49:40 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:40.714920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a156b2-85a0-4293-bb95-ee707b57a186 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:40.816264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a156b2-85a0-4293-bb95-ee707b57a186 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:40.848885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277909d-5b7e-4af5-975b-dea3b6e883b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:40.849489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277909d-5b7e-4af5-975b-dea3b6e883b6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:40.849762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277909d-5b7e-4af5-975b-dea3b6e883b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:40.850029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277909d-5b7e-4af5-975b-dea3b6e883b6 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:40.850649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277909d-5b7e-4af5-975b-dea3b6e883b6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:40.850948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277909d-5b7e-4af5-975b-dea3b6e883b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:40.852323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277909d-5b7e-4af5-975b-dea3b6e883b6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:40.858003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 514/1028] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:40.863578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431c59e-92fc-4958-9999-444271002fbc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:40.976345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431c59e-92fc-4958-9999-444271002fbc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:41.003146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:41.011974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eabbf-a95c-4c1d-a571-556dd21708c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:41.012609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eabbf-a95c-4c1d-a571-556dd21708c4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:41.012885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eabbf-a95c-4c1d-a571-556dd21708c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:41.013151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eabbf-a95c-4c1d-a571-556dd21708c4 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:41.013821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eabbf-a95c-4c1d-a571-556dd21708c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:41.014123 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eabbf-a95c-4c1d-a571-556dd21708c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:41.015572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eabbf-a95c-4c1d-a571-556dd21708c4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:41.020571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 515/1029] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:41.026149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277909d-5b7e-4af5-975b-dea3b6e883b6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:41.029538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:41.133052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277909d-5b7e-4af5-975b-dea3b6e883b6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:41.166963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ff946-349a-4609-b45d-cfe596b65090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:41.167594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ff946-349a-4609-b45d-cfe596b65090 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:41.168526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ff946-349a-4609-b45d-cfe596b65090 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:41.168725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ff946-349a-4609-b45d-cfe596b65090 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:41.169380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ff946-349a-4609-b45d-cfe596b65090 None admin] RBAC: Adding query filter params (interface=public, service_id=ec3b790d72f34e20b8e50f4f7748dab5, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:41.169683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ff946-349a-4609-b45d-cfe596b65090 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:41.171126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ff946-349a-4609-b45d-cfe596b65090 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:41.176260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 515/1030] 10.222.0.86 () {56 vars in 1251 bytes} [Mon Aug 29 18:49:41 2022] GET /identity/v3/endpoints?service_id=ec3b790d72f34e20b8e50f4f7748dab5&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:41.181746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eabbf-a95c-4c1d-a571-556dd21708c4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:41.282686 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eabbf-a95c-4c1d-a571-556dd21708c4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:41.316369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de900f66-70d5-4c6d-a8c9-d91f0bcef3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:41.316973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de900f66-70d5-4c6d-a8c9-d91f0bcef3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:41.317249 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de900f66-70d5-4c6d-a8c9-d91f0bcef3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:41.317517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de900f66-70d5-4c6d-a8c9-d91f0bcef3e4 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:41.318114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de900f66-70d5-4c6d-a8c9-d91f0bcef3e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:41.319801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de900f66-70d5-4c6d-a8c9-d91f0bcef3e4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:41.327059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 516/1031] 10.222.0.86 () {56 vars in 1087 bytes} [Mon Aug 29 18:49:41 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:49:41.465789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532042d-915f-44b1-9e48-8d0cc41f63f0 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:41.465789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532042d-915f-44b1-9e48-8d0cc41f63f0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:41.466204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532042d-915f-44b1-9e48-8d0cc41f63f0 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:41.468448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 516/1032] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:41.474129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089907a-cdee-4c83-83fd-8da8113ac649 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:41.474421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089907a-cdee-4c83-83fd-8da8113ac649 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:41.474698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089907a-cdee-4c83-83fd-8da8113ac649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:41.493368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5089907a-cdee-4c83-83fd-8da8113ac649 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:41.494568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5089907a-cdee-4c83-83fd-8da8113ac649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:41.548499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:41.567892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 517/1033] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:41.839704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532042d-915f-44b1-9e48-8d0cc41f63f0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:41.909443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:41.952446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532042d-915f-44b1-9e48-8d0cc41f63f0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:41.990045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975425b2-7c06-4ce8-b024-5c00812c8514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:41.990678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975425b2-7c06-4ce8-b024-5c00812c8514 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:41.990958 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975425b2-7c06-4ce8-b024-5c00812c8514 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:41.991256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975425b2-7c06-4ce8-b024-5c00812c8514 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:42.002624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975425b2-7c06-4ce8-b024-5c00812c8514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-975425b2-7c06-4ce8-b024-5c00812c8514 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:42.007512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:42.008500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:42.009344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:42.011347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 517/1034] 10.222.0.86 () {56 vars in 1120 bytes} [Mon Aug 29 18:49:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:42.015200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089907a-cdee-4c83-83fd-8da8113ac649 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:42.115819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089907a-cdee-4c83-83fd-8da8113ac649 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:42.149735 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede1709-4008-448f-84ee-d699b6a7e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:42.150348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede1709-4008-448f-84ee-d699b6a7e0ec None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:42.150630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede1709-4008-448f-84ee-d699b6a7e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:42.150905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede1709-4008-448f-84ee-d699b6a7e0ec None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:42.151610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede1709-4008-448f-84ee-d699b6a7e0ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:42.151916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede1709-4008-448f-84ee-d699b6a7e0ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:42.153309 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede1709-4008-448f-84ee-d699b6a7e0ec None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:42.161396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 518/1035] 10.222.0.86 () {56 vars in 1115 bytes} [Mon Aug 29 18:49:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:42.166855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975425b2-7c06-4ce8-b024-5c00812c8514 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:42.276091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975425b2-7c06-4ce8-b024-5c00812c8514 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:42.303453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:42.315134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e6f37-ee5f-4a66-9aaa-1cb69cb4fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:42.315776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e6f37-ee5f-4a66-9aaa-1cb69cb4fc6d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:42.316055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e6f37-ee5f-4a66-9aaa-1cb69cb4fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:42.316324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e6f37-ee5f-4a66-9aaa-1cb69cb4fc6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:42.328315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e6f37-ee5f-4a66-9aaa-1cb69cb4fc6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:42.329806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e6f37-ee5f-4a66-9aaa-1cb69cb4fc6d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ad1e6f37-ee5f-4a66-9aaa-1cb69cb4fc6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:42.344473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:42.345518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:42.346421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:42.346421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:42.346421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:42.346421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 518/1036] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:42.352737 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede1709-4008-448f-84ee-d699b6a7e0ec None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:42.391763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6c1c0-9e8d-44ea-9d98-5951a39f7ae5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:42.391960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6c1c0-9e8d-44ea-9d98-5951a39f7ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:42.392217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6c1c0-9e8d-44ea-9d98-5951a39f7ae5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:42.393576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 519/1037] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:42.398429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27de20-08c7-40ed-a461-3aa14d93a1fd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:42.398710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27de20-08c7-40ed-a461-3aa14d93a1fd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:42.398978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27de20-08c7-40ed-a461-3aa14d93a1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:42.414328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:42.419496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-be27de20-08c7-40ed-a461-3aa14d93a1fd None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:42.420934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-be27de20-08c7-40ed-a461-3aa14d93a1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:42.474237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede1709-4008-448f-84ee-d699b6a7e0ec None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:42.492506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 520/1038] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:42.510000 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab2cd9-9f15-4115-9c81-9d5175096fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:42.510836 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2cd9-9f15-4115-9c81-9d5175096fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:42.511326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2cd9-9f15-4115-9c81-9d5175096fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:42.511770 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2cd9-9f15-4115-9c81-9d5175096fb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:42.512605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab2cd9-9f15-4115-9c81-9d5175096fb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:42.513092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab2cd9-9f15-4115-9c81-9d5175096fb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:42.514738 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab2cd9-9f15-4115-9c81-9d5175096fb9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:42.526942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 519/1039] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:42.532722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27de20-08c7-40ed-a461-3aa14d93a1fd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:42.634115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27de20-08c7-40ed-a461-3aa14d93a1fd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:42.643684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:42.671891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea4a26-238f-419d-9240-94fabb09462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:42.672526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea4a26-238f-419d-9240-94fabb09462a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:42.672805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea4a26-238f-419d-9240-94fabb09462a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:42.673075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea4a26-238f-419d-9240-94fabb09462a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:42.687636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea4a26-238f-419d-9240-94fabb09462a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:42.689747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea4a26-238f-419d-9240-94fabb09462a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4aea4a26-238f-419d-9240-94fabb09462a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:42.702519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:42.703560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:42.704740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:42.704740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:42.704740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:42.704740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:42.704740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:42.704740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:42.704740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:42.704740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:42.704740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:42.704740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:42.704740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 521/1040] 10.222.0.86 () {56 vars in 1093 bytes} [Mon Aug 29 18:49:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:42.711978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab2cd9-9f15-4115-9c81-9d5175096fb9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:42.803311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea4a26-238f-419d-9240-94fabb09462a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:42.831297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab2cd9-9f15-4115-9c81-9d5175096fb9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:42.841228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:42.867488 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb1d70-9c0d-41e8-a89c-b621a5a6eb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:42.868108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb1d70-9c0d-41e8-a89c-b621a5a6eb46 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:42.868383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb1d70-9c0d-41e8-a89c-b621a5a6eb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:42.868654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb1d70-9c0d-41e8-a89c-b621a5a6eb46 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:42.869373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb1d70-9c0d-41e8-a89c-b621a5a6eb46 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:42.869675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb1d70-9c0d-41e8-a89c-b621a5a6eb46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:42.871143 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb1d70-9c0d-41e8-a89c-b621a5a6eb46 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:42.889212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 520/1041] 10.222.0.86 () {56 vars in 1133 bytes} [Mon Aug 29 18:49:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:42.894822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb1d70-9c0d-41e8-a89c-b621a5a6eb46 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:42.912135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea4a26-238f-419d-9240-94fabb09462a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:42.947840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a463f79-6ab9-48aa-af7e-530cb2cff378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:42.948503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a463f79-6ab9-48aa-af7e-530cb2cff378 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:42.948783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a463f79-6ab9-48aa-af7e-530cb2cff378 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:42.949591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a463f79-6ab9-48aa-af7e-530cb2cff378 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:42.961451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a463f79-6ab9-48aa-af7e-530cb2cff378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2a463f79-6ab9-48aa-af7e-530cb2cff378 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:42.966830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:42.967873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:42.968779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:42.973043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 522/1042] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:49:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:42.973787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a463f79-6ab9-48aa-af7e-530cb2cff378 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:43.002817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:43.004455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb1d70-9c0d-41e8-a89c-b621a5a6eb46 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:43.041343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f91969-61e0-4948-bcc1-6b040528677a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:43.042253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f91969-61e0-4948-bcc1-6b040528677a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:43.042761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f91969-61e0-4948-bcc1-6b040528677a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:43.043080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f91969-61e0-4948-bcc1-6b040528677a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:43.082145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:43.094881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a463f79-6ab9-48aa-af7e-530cb2cff378 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:43.122521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f91969-61e0-4948-bcc1-6b040528677a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:43.124531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f91969-61e0-4948-bcc1-6b040528677a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:43.140216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28b717-8e44-4b06-a9c5-c2189e684361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:43.141814 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28b717-8e44-4b06-a9c5-c2189e684361 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:43.142408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28b717-8e44-4b06-a9c5-c2189e684361 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:43.143152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28b717-8e44-4b06-a9c5-c2189e684361 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:43.144269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28b717-8e44-4b06-a9c5-c2189e684361 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:43.145188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28b717-8e44-4b06-a9c5-c2189e684361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:43.146781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28b717-8e44-4b06-a9c5-c2189e684361 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:43.159269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 523/1043] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:49:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:43.164952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28b717-8e44-4b06-a9c5-c2189e684361 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:43.233444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 521/1044] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:42 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 18:49:43.239137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f91969-61e0-4948-bcc1-6b040528677a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:43.266095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:43.277329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28b717-8e44-4b06-a9c5-c2189e684361 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:43.309399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:43.312497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c036ed-3aa7-4648-854a-36599abb0339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:43.313204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c036ed-3aa7-4648-854a-36599abb0339 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:43.313479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c036ed-3aa7-4648-854a-36599abb0339 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:43.313748 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c036ed-3aa7-4648-854a-36599abb0339 None admin] PATH_INFO: `/v3/users/890c1a918b88488fb67e6ad7d3ec816a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:43.328048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c036ed-3aa7-4648-854a-36599abb0339 None admin] RBAC: Authorizing `identity:get_user(user_id=890c1a918b88488fb67e6ad7d3ec816a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:43.329466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c036ed-3aa7-4648-854a-36599abb0339 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:43.331169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 524/1045] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:49:43 2022] GET /identity/v3/users/890c1a918b88488fb67e6ad7d3ec816a => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:43.332306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6873d5-85b3-47cf-b62e-c8678f8610e3 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:43.332584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6873d5-85b3-47cf-b62e-c8678f8610e3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:43.332856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6873d5-85b3-47cf-b62e-c8678f8610e3 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:43.334032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 525/1046] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:43.337317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6873d5-85b3-47cf-b62e-c8678f8610e3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:43.348036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f91969-61e0-4948-bcc1-6b040528677a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:43.382283 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d090e66-48f9-449c-b050-279910c781ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:43.382908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090e66-48f9-449c-b050-279910c781ff None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:43.383205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090e66-48f9-449c-b050-279910c781ff None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:43.383474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090e66-48f9-449c-b050-279910c781ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:43.393675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d090e66-48f9-449c-b050-279910c781ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:43.395142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d090e66-48f9-449c-b050-279910c781ff None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5d090e66-48f9-449c-b050-279910c781ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:43.414334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:43.415348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:43.416251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:43.416251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:43.416251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:43.416251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 522/1047] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:43.417807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1889d77-de9c-4a86-b501-3ba2cf0b89ca None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:43.418219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1889d77-de9c-4a86-b501-3ba2cf0b89ca None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:43.418616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1889d77-de9c-4a86-b501-3ba2cf0b89ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:43.441922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6873d5-85b3-47cf-b62e-c8678f8610e3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:43.442887 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a1889d77-de9c-4a86-b501-3ba2cf0b89ca None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:43.444877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a1889d77-de9c-4a86-b501-3ba2cf0b89ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:43.483487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce838608-7b39-402f-b842-5fd653e7e181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:43.485670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce838608-7b39-402f-b842-5fd653e7e181 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:43.485954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce838608-7b39-402f-b842-5fd653e7e181 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:43.486225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce838608-7b39-402f-b842-5fd653e7e181 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:43.506379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:43.525406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 523/1048] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:43.526262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1889d77-de9c-4a86-b501-3ba2cf0b89ca None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:43.564294 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce838608-7b39-402f-b842-5fd653e7e181 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:43.565861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce838608-7b39-402f-b842-5fd653e7e181 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:43.580844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:43.634420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1889d77-de9c-4a86-b501-3ba2cf0b89ca None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:43.670184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b77e4-1913-4c40-9183-c75e8df62092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:43.670805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b77e4-1913-4c40-9183-c75e8df62092 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:43.671121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b77e4-1913-4c40-9183-c75e8df62092 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:43.671401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b77e4-1913-4c40-9183-c75e8df62092 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:43.672039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b77e4-1913-4c40-9183-c75e8df62092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:43.672344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b77e4-1913-4c40-9183-c75e8df62092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:43.678133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 526/1049] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:43 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:49:43.678295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b77e4-1913-4c40-9183-c75e8df62092 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:43.688030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce838608-7b39-402f-b842-5fd653e7e181 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:43.692067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 524/1050] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:43.697883 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b77e4-1913-4c40-9183-c75e8df62092 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:43.748932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:43.800500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce838608-7b39-402f-b842-5fd653e7e181 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:43.805875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b77e4-1913-4c40-9183-c75e8df62092 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:43.817904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:43.834849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b21291-1e6b-4d36-8065-e155554ae6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:43.835510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b21291-1e6b-4d36-8065-e155554ae6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:43.835791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b21291-1e6b-4d36-8065-e155554ae6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:43.836057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b21291-1e6b-4d36-8065-e155554ae6b9 None admin] PATH_INFO: `/v3/projects/9d41aad68c264e76adc4449c056a869b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:43.837153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b21291-1e6b-4d36-8065-e155554ae6b9 None admin] RBAC: Authorizing `identity:get_project(project_id=9d41aad68c264e76adc4449c056a869b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:43.838560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b21291-1e6b-4d36-8065-e155554ae6b9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:43.840482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 527/1051] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:49:43 2022] GET /identity/v3/projects/9d41aad68c264e76adc4449c056a869b => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:43.841132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b21291-1e6b-4d36-8065-e155554ae6b9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:43.854277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16cb1c-8139-4d50-8192-194b10de2a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:43.854919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16cb1c-8139-4d50-8192-194b10de2a36 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:43.855812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16cb1c-8139-4d50-8192-194b10de2a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:43.856087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16cb1c-8139-4d50-8192-194b10de2a36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:43.863955 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16cb1c-8139-4d50-8192-194b10de2a36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:43.865418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16cb1c-8139-4d50-8192-194b10de2a36 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8a16cb1c-8139-4d50-8192-194b10de2a36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:43.878003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:43.879247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:43.880397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:43.880397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:43.880397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:43.880397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:43.880397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:43.880397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:43.880397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:43.880397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:43.880397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 525/1052] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:43.880397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16cb1c-8139-4d50-8192-194b10de2a36 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:43.949103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b21291-1e6b-4d36-8065-e155554ae6b9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:43.965774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:43.979428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16cb1c-8139-4d50-8192-194b10de2a36 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:43.984124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce2dd2-ad3e-4745-9f68-603925ecb527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:43.984773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce2dd2-ad3e-4745-9f68-603925ecb527 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:43.985049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce2dd2-ad3e-4745-9f68-603925ecb527 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:43.985315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce2dd2-ad3e-4745-9f68-603925ecb527 None admin] PATH_INFO: `/v3/services/image` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:43.985887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce2dd2-ad3e-4745-9f68-603925ecb527 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:43.987266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce2dd2-ad3e-4745-9f68-603925ecb527 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-44ce2dd2-ad3e-4745-9f68-603925ecb527 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:43.995931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:43.997061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:43.998264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 528/1053] 10.222.0.86 () {56 vars in 1105 bytes} [Mon Aug 29 18:49:43 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:43.999277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce2dd2-ad3e-4745-9f68-603925ecb527 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:44.016037 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d08581-fbe7-49da-8337-87a0a3845572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:44.016653 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d08581-fbe7-49da-8337-87a0a3845572 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:44.016934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d08581-fbe7-49da-8337-87a0a3845572 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:44.017210 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d08581-fbe7-49da-8337-87a0a3845572 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:44.017932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d08581-fbe7-49da-8337-87a0a3845572 None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=9d41aad68c264e76adc4449c056a869b, user.id=890c1a918b88488fb67e6ad7d3ec816a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:44.018242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d08581-fbe7-49da-8337-87a0a3845572 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:44.019700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d08581-fbe7-49da-8337-87a0a3845572 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:44.024995 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 526/1054] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:43 2022] GET /identity/v3/role_assignments?user.id=890c1a918b88488fb67e6ad7d3ec816a&scope.project.id=9d41aad68c264e76adc4449c056a869b&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:44.025762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d08581-fbe7-49da-8337-87a0a3845572 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:44.063113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:44.122763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce2dd2-ad3e-4745-9f68-603925ecb527 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:44.141451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d08581-fbe7-49da-8337-87a0a3845572 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:44.161066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ee7cb-98ea-4973-9e3e-96765bdab50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:44.161671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ee7cb-98ea-4973-9e3e-96765bdab50a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:44.161955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ee7cb-98ea-4973-9e3e-96765bdab50a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:44.162687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ee7cb-98ea-4973-9e3e-96765bdab50a None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:44.162903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ee7cb-98ea-4973-9e3e-96765bdab50a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:44.163246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ee7cb-98ea-4973-9e3e-96765bdab50a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:44.164674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ee7cb-98ea-4973-9e3e-96765bdab50a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:44.178584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 529/1055] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:44.185211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ee7cb-98ea-4973-9e3e-96765bdab50a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:44.186879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73800da4-6539-4c9d-a5fc-9bb7f5ae1cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:44.187586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73800da4-6539-4c9d-a5fc-9bb7f5ae1cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:44.187999 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73800da4-6539-4c9d-a5fc-9bb7f5ae1cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:44.188257 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73800da4-6539-4c9d-a5fc-9bb7f5ae1cc5 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:44.189538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73800da4-6539-4c9d-a5fc-9bb7f5ae1cc5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:44.189538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73800da4-6539-4c9d-a5fc-9bb7f5ae1cc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:44.190712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73800da4-6539-4c9d-a5fc-9bb7f5ae1cc5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:44.200868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 527/1056] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:44.201705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73800da4-6539-4c9d-a5fc-9bb7f5ae1cc5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:44.231086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:44.298126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ee7cb-98ea-4973-9e3e-96765bdab50a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:44.302564 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:44.309078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73800da4-6539-4c9d-a5fc-9bb7f5ae1cc5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:44.336406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb71ee0-7ef0-47ac-9a0d-a840e052862a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:44.337011 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb71ee0-7ef0-47ac-9a0d-a840e052862a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:44.337289 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb71ee0-7ef0-47ac-9a0d-a840e052862a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:44.337553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb71ee0-7ef0-47ac-9a0d-a840e052862a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:44.338269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb71ee0-7ef0-47ac-9a0d-a840e052862a None admin] RBAC: Adding query filter params (role.id=4ee6c8fb1f294eb0824448e1f26984d6, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=045b22d046c549f2b53d2ed73b94f936) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:44.338571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb71ee0-7ef0-47ac-9a0d-a840e052862a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:44.340612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb71ee0-7ef0-47ac-9a0d-a840e052862a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:44.346448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 530/1057] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:44 2022] GET /identity/v3/role_assignments?user.id=045b22d046c549f2b53d2ed73b94f936&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=4ee6c8fb1f294eb0824448e1f26984d6 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:44.347451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea67ec-3875-4be0-bba6-0db3be707ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:44.348063 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea67ec-3875-4be0-bba6-0db3be707ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:44.348340 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea67ec-3875-4be0-bba6-0db3be707ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:44.348610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea67ec-3875-4be0-bba6-0db3be707ae6 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:44.349243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea67ec-3875-4be0-bba6-0db3be707ae6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:44.349547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea67ec-3875-4be0-bba6-0db3be707ae6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:44.350932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea67ec-3875-4be0-bba6-0db3be707ae6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:44.357433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 528/1058] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:44.363063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb71ee0-7ef0-47ac-9a0d-a840e052862a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:44.481221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb71ee0-7ef0-47ac-9a0d-a840e052862a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:44.483763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:44.521715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468de9c-1eba-4850-89fb-50a844740ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:44.522325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468de9c-1eba-4850-89fb-50a844740ce7 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:44.522604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468de9c-1eba-4850-89fb-50a844740ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:44.522874 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468de9c-1eba-4850-89fb-50a844740ce7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:44.523638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468de9c-1eba-4850-89fb-50a844740ce7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:44.525006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468de9c-1eba-4850-89fb-50a844740ce7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:44.547030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 531/1059] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 18:49:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:49:45.936925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c70b0-115f-4d20-b9cd-eb9be2cfbd33 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:45.937437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c70b0-115f-4d20-b9cd-eb9be2cfbd33 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:45.937514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c70b0-115f-4d20-b9cd-eb9be2cfbd33 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:45.939048 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 529/1060] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:45.944049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5354c55-6ff2-4b22-9390-5ebf5ec09f8b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:45.944702 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5354c55-6ff2-4b22-9390-5ebf5ec09f8b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:45.944861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5354c55-6ff2-4b22-9390-5ebf5ec09f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:45.963352 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c5354c55-6ff2-4b22-9390-5ebf5ec09f8b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:45.964581 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c5354c55-6ff2-4b22-9390-5ebf5ec09f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:46.040546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 532/1061] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:46.249489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99065fc0-abdc-49d5-af5c-e58c5a2e6e40 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:46.249909 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99065fc0-abdc-49d5-af5c-e58c5a2e6e40 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:46.250265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99065fc0-abdc-49d5-af5c-e58c5a2e6e40 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:46.251998 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 530/1062] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:46.256994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185859b7-0ac1-4e26-b9e9-e40ac3aa9877 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:46.257440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185859b7-0ac1-4e26-b9e9-e40ac3aa9877 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:46.257882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185859b7-0ac1-4e26-b9e9-e40ac3aa9877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:46.276217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-185859b7-0ac1-4e26-b9e9-e40ac3aa9877 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:46.278117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-185859b7-0ac1-4e26-b9e9-e40ac3aa9877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:46.308379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99065fc0-abdc-49d5-af5c-e58c5a2e6e40 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:46.350043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 533/1063] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:46.420938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99065fc0-abdc-49d5-af5c-e58c5a2e6e40 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:46.454859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f7708-dcac-4b2c-9a1e-38919cb23a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:46.455518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f7708-dcac-4b2c-9a1e-38919cb23a99 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:46.455803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f7708-dcac-4b2c-9a1e-38919cb23a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:46.456078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f7708-dcac-4b2c-9a1e-38919cb23a99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:46.460447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6849826-637b-4452-9711-7ac157d372f8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:46.460731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6849826-637b-4452-9711-7ac157d372f8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:46.461027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6849826-637b-4452-9711-7ac157d372f8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:46.462476 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 534/1064] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:46.467703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f7708-dcac-4b2c-9a1e-38919cb23a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:46.467915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cc9d1-b552-4b1c-b5bd-cf2f2e2ada42 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:46.468169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cc9d1-b552-4b1c-b5bd-cf2f2e2ada42 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:46.468440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cc9d1-b552-4b1c-b5bd-cf2f2e2ada42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b83f7708-dcac-4b2c-9a1e-38919cb23a99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:46.472471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:46.473393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:46.474266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:46.475336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 531/1065] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:49:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:46.479809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f7708-dcac-4b2c-9a1e-38919cb23a99 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:46.490896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f68cc9d1-b552-4b1c-b5bd-cf2f2e2ada42 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:46.492492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f68cc9d1-b552-4b1c-b5bd-cf2f2e2ada42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:46.502483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:46.506333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:46.569087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 535/1066] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:46.586573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f7708-dcac-4b2c-9a1e-38919cb23a99 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:46.621814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f80ba-b79d-4a2f-98f1-8db0eb944231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:46.622419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f80ba-b79d-4a2f-98f1-8db0eb944231 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:46.622639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cc9d1-b552-4b1c-b5bd-cf2f2e2ada42 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:46.622769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f80ba-b79d-4a2f-98f1-8db0eb944231 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:46.622960 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f80ba-b79d-4a2f-98f1-8db0eb944231 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:46.623731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f80ba-b79d-4a2f-98f1-8db0eb944231 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:46.624036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f80ba-b79d-4a2f-98f1-8db0eb944231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:46.625403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f80ba-b79d-4a2f-98f1-8db0eb944231 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:46.634682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 532/1067] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:49:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:46.641556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f80ba-b79d-4a2f-98f1-8db0eb944231 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:46.728449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:46.744769 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cc9d1-b552-4b1c-b5bd-cf2f2e2ada42 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:46.747171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f80ba-b79d-4a2f-98f1-8db0eb944231 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:46.784879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9772203-3ab2-4a43-a54d-41a877dde844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:46.785212 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cd24a-20ff-4a33-ac35-73347922d3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:46.785513 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772203-3ab2-4a43-a54d-41a877dde844 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:46.785794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772203-3ab2-4a43-a54d-41a877dde844 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:46.785911 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cd24a-20ff-4a33-ac35-73347922d3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:46.786068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772203-3ab2-4a43-a54d-41a877dde844 None admin] PATH_INFO: `/v3/users/890c1a918b88488fb67e6ad7d3ec816a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:46.786161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cd24a-20ff-4a33-ac35-73347922d3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:46.786349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cd24a-20ff-4a33-ac35-73347922d3a0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:46.787234 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9772203-3ab2-4a43-a54d-41a877dde844 None admin] RBAC: Authorizing `identity:get_user(user_id=890c1a918b88488fb67e6ad7d3ec816a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:46.788692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9772203-3ab2-4a43-a54d-41a877dde844 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:46.790471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 533/1068] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:49:46 2022] GET /identity/v3/users/890c1a918b88488fb67e6ad7d3ec816a => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:46.795950 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9772203-3ab2-4a43-a54d-41a877dde844 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:46.797172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cd24a-20ff-4a33-ac35-73347922d3a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8a1cd24a-20ff-4a33-ac35-73347922d3a0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:46.803107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:46.804027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:46.805009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 536/1069] 10.222.0.86 () {56 vars in 1120 bytes} [Mon Aug 29 18:49:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:46.810366 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cd24a-20ff-4a33-ac35-73347922d3a0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:46.820260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:46.899271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9772203-3ab2-4a43-a54d-41a877dde844 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:46.910675 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cd24a-20ff-4a33-ac35-73347922d3a0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:46.937691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c221f-a283-4ed9-8409-53d1f25d4958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:46.938412 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c221f-a283-4ed9-8409-53d1f25d4958 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:46.938575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c221f-a283-4ed9-8409-53d1f25d4958 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:46.938842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c221f-a283-4ed9-8409-53d1f25d4958 None admin] PATH_INFO: `/v3/projects/9d41aad68c264e76adc4449c056a869b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:46.940006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c221f-a283-4ed9-8409-53d1f25d4958 None admin] RBAC: Authorizing `identity:get_project(project_id=9d41aad68c264e76adc4449c056a869b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:46.941421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c221f-a283-4ed9-8409-53d1f25d4958 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:46.943065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 534/1070] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:49:46 2022] GET /identity/v3/projects/9d41aad68c264e76adc4449c056a869b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:46.943821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c221f-a283-4ed9-8409-53d1f25d4958 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:46.953868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dcd3d-1137-478a-8726-14d43e3bb8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:46.954481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dcd3d-1137-478a-8726-14d43e3bb8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:46.954761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dcd3d-1137-478a-8726-14d43e3bb8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:46.955056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dcd3d-1137-478a-8726-14d43e3bb8e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:46.955727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dcd3d-1137-478a-8726-14d43e3bb8e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:46.956030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dcd3d-1137-478a-8726-14d43e3bb8e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:46.957392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dcd3d-1137-478a-8726-14d43e3bb8e2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:46.962590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:46.967224 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 537/1071] 10.222.0.86 () {56 vars in 1115 bytes} [Mon Aug 29 18:49:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:46.968008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dcd3d-1137-478a-8726-14d43e3bb8e2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:47.034457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:47.045225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c221f-a283-4ed9-8409-53d1f25d4958 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:47.069651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dcd3d-1137-478a-8726-14d43e3bb8e2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:47.081812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6083fc-4951-4f53-a7f0-92966968ea12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:47.082418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6083fc-4951-4f53-a7f0-92966968ea12 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:47.082694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6083fc-4951-4f53-a7f0-92966968ea12 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:47.082965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6083fc-4951-4f53-a7f0-92966968ea12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:47.102424 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d32e8-58d5-4632-973f-808a87a6819b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:47.103095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d32e8-58d5-4632-973f-808a87a6819b None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:47.103726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d32e8-58d5-4632-973f-808a87a6819b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:47.103726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d32e8-58d5-4632-973f-808a87a6819b None admin] PATH_INFO: `/v3/projects/9d41aad68c264e76adc4449c056a869b/users/890c1a918b88488fb67e6ad7d3ec816a/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:47.105560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d32e8-58d5-4632-973f-808a87a6819b None admin] RBAC: Authorizing `identity:create_grant(project_id=9d41aad68c264e76adc4449c056a869b, user_id=890c1a918b88488fb67e6ad7d3ec816a, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:47.106976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d32e8-58d5-4632-973f-808a87a6819b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:47.119398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 538/1072] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:49:46 2022] PUT /identity/v3/projects/9d41aad68c264e76adc4449c056a869b/users/890c1a918b88488fb67e6ad7d3ec816a/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:49:47.120189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d32e8-58d5-4632-973f-808a87a6819b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:47.152812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6083fc-4951-4f53-a7f0-92966968ea12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:47.154292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6083fc-4951-4f53-a7f0-92966968ea12 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:47.174645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:47.250314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d32e8-58d5-4632-973f-808a87a6819b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:47.284097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 535/1073] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:49:47.285618 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:47.291831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6083fc-4951-4f53-a7f0-92966968ea12 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:47.295524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadf69b-2959-48ec-9dfc-90e8fdb367d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:47.296106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadf69b-2959-48ec-9dfc-90e8fdb367d4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:47.296392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadf69b-2959-48ec-9dfc-90e8fdb367d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:47.296656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadf69b-2959-48ec-9dfc-90e8fdb367d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:47.307784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadf69b-2959-48ec-9dfc-90e8fdb367d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:47.309756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadf69b-2959-48ec-9dfc-90e8fdb367d4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-aaadf69b-2959-48ec-9dfc-90e8fdb367d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:47.327684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:47.328684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:47.329596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:47.329596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:47.329596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:47.329596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 539/1074] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:47.334725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadf69b-2959-48ec-9dfc-90e8fdb367d4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:47.399081 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6083fc-4951-4f53-a7f0-92966968ea12 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:47.418328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:47.437072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7b8fe-64d2-4ab1-94ee-ed3aba8865a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:47.437705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7b8fe-64d2-4ab1-94ee-ed3aba8865a0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:47.437986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7b8fe-64d2-4ab1-94ee-ed3aba8865a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:47.438261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7b8fe-64d2-4ab1-94ee-ed3aba8865a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:47.443141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadf69b-2959-48ec-9dfc-90e8fdb367d4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:47.449175 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7b8fe-64d2-4ab1-94ee-ed3aba8865a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:47.450621 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7b8fe-64d2-4ab1-94ee-ed3aba8865a0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-92f7b8fe-64d2-4ab1-94ee-ed3aba8865a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:47.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:47.469861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:47.471982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:47.471982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:47.471982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:47.474589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 536/1075] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:47.475569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7b8fe-64d2-4ab1-94ee-ed3aba8865a0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:47.486967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003c700-dcec-4de5-8c46-5d8c645d375d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:47.487976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003c700-dcec-4de5-8c46-5d8c645d375d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:47.488258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003c700-dcec-4de5-8c46-5d8c645d375d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:47.488531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003c700-dcec-4de5-8c46-5d8c645d375d None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:47.489538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003c700-dcec-4de5-8c46-5d8c645d375d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:47.489646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003c700-dcec-4de5-8c46-5d8c645d375d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:47.490975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003c700-dcec-4de5-8c46-5d8c645d375d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:47.505348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 540/1076] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:47.510969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003c700-dcec-4de5-8c46-5d8c645d375d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:47.565633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:47.590951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7b8fe-64d2-4ab1-94ee-ed3aba8865a0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:47.629004 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f04f92-5a4b-4162-b596-b050cefe40e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:47.629801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f04f92-5a4b-4162-b596-b050cefe40e8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:47.630082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f04f92-5a4b-4162-b596-b050cefe40e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:47.630356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f04f92-5a4b-4162-b596-b050cefe40e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:47.631039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f04f92-5a4b-4162-b596-b050cefe40e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:47.631350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f04f92-5a4b-4162-b596-b050cefe40e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:47.632742 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f04f92-5a4b-4162-b596-b050cefe40e8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:47.642439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003c700-dcec-4de5-8c46-5d8c645d375d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:47.655029 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 537/1077] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:47.693558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c03d5-6071-425f-b472-aff9d5af4f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:47.694459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c03d5-6071-425f-b472-aff9d5af4f62 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:47.694741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c03d5-6071-425f-b472-aff9d5af4f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:47.695023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c03d5-6071-425f-b472-aff9d5af4f62 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:47.703592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c03d5-6071-425f-b472-aff9d5af4f62 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:47.705106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c03d5-6071-425f-b472-aff9d5af4f62 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8f5c03d5-6071-425f-b472-aff9d5af4f62 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:47.718524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:47.719743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:47.720896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:47.720896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:47.720896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:47.720896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:47.720896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:47.720896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:47.720896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:47.720896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:47.720896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:47.720896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:47.720896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 541/1078] 10.222.0.86 () {56 vars in 1093 bytes} [Mon Aug 29 18:49:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:47.725707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f04f92-5a4b-4162-b596-b050cefe40e8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:47.758339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:47.838217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f04f92-5a4b-4162-b596-b050cefe40e8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:47.874389 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570b47e-d6fe-4448-b911-c7de124865b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:47.875021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570b47e-d6fe-4448-b911-c7de124865b5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:47.875293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570b47e-d6fe-4448-b911-c7de124865b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:47.875561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570b47e-d6fe-4448-b911-c7de124865b5 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:47.876286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570b47e-d6fe-4448-b911-c7de124865b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:47.876587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570b47e-d6fe-4448-b911-c7de124865b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:47.878046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570b47e-d6fe-4448-b911-c7de124865b5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:47.895640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 538/1079] 10.222.0.86 () {56 vars in 1133 bytes} [Mon Aug 29 18:49:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:47.901579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c03d5-6071-425f-b472-aff9d5af4f62 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:48.009841 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:48.013144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c03d5-6071-425f-b472-aff9d5af4f62 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:48.047878 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a80dd6-8e30-4959-8642-7ab1be3a8014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:48.048496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a80dd6-8e30-4959-8642-7ab1be3a8014 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:48.048771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a80dd6-8e30-4959-8642-7ab1be3a8014 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:48.049035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a80dd6-8e30-4959-8642-7ab1be3a8014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:48.059224 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a80dd6-8e30-4959-8642-7ab1be3a8014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:48.060641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a80dd6-8e30-4959-8642-7ab1be3a8014 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d1a80dd6-8e30-4959-8642-7ab1be3a8014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:48.075102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:48.076028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:48.077083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:48.077083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:48.077083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:48.077083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 542/1080] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:48.083341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570b47e-d6fe-4448-b911-c7de124865b5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:48.169936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:48.182115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570b47e-d6fe-4448-b911-c7de124865b5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:48.215506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5df26-a1c8-46ba-ade0-75a1be7ae0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:48.216102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5df26-a1c8-46ba-ade0-75a1be7ae0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:48.216377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5df26-a1c8-46ba-ade0-75a1be7ae0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:48.216643 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5df26-a1c8-46ba-ade0-75a1be7ae0d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:48.217359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5df26-a1c8-46ba-ade0-75a1be7ae0d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:48.217654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5df26-a1c8-46ba-ade0-75a1be7ae0d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:48.219013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5df26-a1c8-46ba-ade0-75a1be7ae0d0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:48.235732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 539/1081] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:48.236949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a80dd6-8e30-4959-8642-7ab1be3a8014 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:48.334282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a80dd6-8e30-4959-8642-7ab1be3a8014 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:48.366462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae361699-aded-4006-aaad-ac9a6b73f5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:48.367103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae361699-aded-4006-aaad-ac9a6b73f5ec None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:48.367383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae361699-aded-4006-aaad-ac9a6b73f5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:48.367650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae361699-aded-4006-aaad-ac9a6b73f5ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:48.374050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae361699-aded-4006-aaad-ac9a6b73f5ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:48.375507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae361699-aded-4006-aaad-ac9a6b73f5ec None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ae361699-aded-4006-aaad-ac9a6b73f5ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:48.386756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:48.387808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:48.393575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:48.393575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:48.393575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:48.393575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:48.393575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:48.393575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:48.393575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:48.393575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:48.393575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 543/1082] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:48.394067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5df26-a1c8-46ba-ade0-75a1be7ae0d0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:48.492518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5df26-a1c8-46ba-ade0-75a1be7ae0d0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:48.525995 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8decaca-f12c-4104-9d96-2f7c860ae2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:48.526595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8decaca-f12c-4104-9d96-2f7c860ae2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:48.526873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8decaca-f12c-4104-9d96-2f7c860ae2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:48.527183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8decaca-f12c-4104-9d96-2f7c860ae2d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:48.527868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8decaca-f12c-4104-9d96-2f7c860ae2d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:48.528171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8decaca-f12c-4104-9d96-2f7c860ae2d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:48.529580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8decaca-f12c-4104-9d96-2f7c860ae2d2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:48.541120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 540/1083] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:48.547125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae361699-aded-4006-aaad-ac9a6b73f5ec None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:48.562721 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:48.646908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae361699-aded-4006-aaad-ac9a6b73f5ec None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:48.679869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e0217-ceae-4b0a-ad1d-27c67a9a2239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:48.680814 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e0217-ceae-4b0a-ad1d-27c67a9a2239 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:48.681092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e0217-ceae-4b0a-ad1d-27c67a9a2239 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:48.681365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e0217-ceae-4b0a-ad1d-27c67a9a2239 None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/045b22d046c549f2b53d2ed73b94f936/roles/4ee6c8fb1f294eb0824448e1f26984d6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:48.684022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e0217-ceae-4b0a-ad1d-27c67a9a2239 None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=045b22d046c549f2b53d2ed73b94f936, role_id=4ee6c8fb1f294eb0824448e1f26984d6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:48.685469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e0217-ceae-4b0a-ad1d-27c67a9a2239 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:48.698567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 544/1084] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:49:48 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/045b22d046c549f2b53d2ed73b94f936/roles/4ee6c8fb1f294eb0824448e1f26984d6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:49:49.087238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde88fb-b97b-4d2a-bf7f-7df0a1fdef62 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:49.087768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde88fb-b97b-4d2a-bf7f-7df0a1fdef62 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:49.087768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde88fb-b97b-4d2a-bf7f-7df0a1fdef62 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:49.089094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 541/1085] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:49.094574 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317b7d9-e6c1-490a-a92f-f31680ae03cb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:49.094986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317b7d9-e6c1-490a-a92f-f31680ae03cb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:49.095219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317b7d9-e6c1-490a-a92f-f31680ae03cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:49.114168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6317b7d9-e6c1-490a-a92f-f31680ae03cb None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:49.115390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6317b7d9-e6c1-490a-a92f-f31680ae03cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:49.185577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 545/1086] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:49.451390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde88fb-b97b-4d2a-bf7f-7df0a1fdef62 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:49.462060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:49.555872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde88fb-b97b-4d2a-bf7f-7df0a1fdef62 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:49.593061 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da4329-f7f4-4f4d-8f5a-67e94e10e0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:49.593688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da4329-f7f4-4f4d-8f5a-67e94e10e0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:49.593963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da4329-f7f4-4f4d-8f5a-67e94e10e0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:49.594231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da4329-f7f4-4f4d-8f5a-67e94e10e0a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:49.604200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da4329-f7f4-4f4d-8f5a-67e94e10e0a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-98da4329-f7f4-4f4d-8f5a-67e94e10e0a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:49.609045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:49.610169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:49.611097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 542/1087] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:49:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:49.616262 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317b7d9-e6c1-490a-a92f-f31680ae03cb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:49.618459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:49.721435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317b7d9-e6c1-490a-a92f-f31680ae03cb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:49.754945 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396739c0-68ec-442a-928c-75b7337c3e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:49.755600 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396739c0-68ec-442a-928c-75b7337c3e2a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:49.755876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396739c0-68ec-442a-928c-75b7337c3e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:49.756144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396739c0-68ec-442a-928c-75b7337c3e2a None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:49.756823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396739c0-68ec-442a-928c-75b7337c3e2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:49.757123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396739c0-68ec-442a-928c-75b7337c3e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:49.758522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396739c0-68ec-442a-928c-75b7337c3e2a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:49.767778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 546/1088] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:49:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:49.773607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da4329-f7f4-4f4d-8f5a-67e94e10e0a1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:49.839520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:49.872685 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da4329-f7f4-4f4d-8f5a-67e94e10e0a1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:49.904956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8875f7-40fb-4cde-b6ad-a818dd6364cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:49.905568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8875f7-40fb-4cde-b6ad-a818dd6364cc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:49.905843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8875f7-40fb-4cde-b6ad-a818dd6364cc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:49.906110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8875f7-40fb-4cde-b6ad-a818dd6364cc None admin] PATH_INFO: `/v3/users/890c1a918b88488fb67e6ad7d3ec816a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:49.907237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8875f7-40fb-4cde-b6ad-a818dd6364cc None admin] RBAC: Authorizing `identity:get_user(user_id=890c1a918b88488fb67e6ad7d3ec816a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:49.908644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8875f7-40fb-4cde-b6ad-a818dd6364cc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:49.910182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 543/1089] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:49:49 2022] GET /identity/v3/users/890c1a918b88488fb67e6ad7d3ec816a => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:49.916160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396739c0-68ec-442a-928c-75b7337c3e2a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:49.976858 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:50.014360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396739c0-68ec-442a-928c-75b7337c3e2a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:50.048096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2bcc-9a01-42bb-84ab-11e36b978b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:50.048697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2bcc-9a01-42bb-84ab-11e36b978b49 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:50.048973 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2bcc-9a01-42bb-84ab-11e36b978b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:50.049240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2bcc-9a01-42bb-84ab-11e36b978b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:50.114520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2bcc-9a01-42bb-84ab-11e36b978b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:50.115979 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2bcc-9a01-42bb-84ab-11e36b978b49 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:50.174888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:50.212035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 547/1090] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 18:49:50.218550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8875f7-40fb-4cde-b6ad-a818dd6364cc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:50.315989 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8875f7-40fb-4cde-b6ad-a818dd6364cc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:50.349432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b560543-53c8-4d76-abc7-286b3dffa548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:50.350054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b560543-53c8-4d76-abc7-286b3dffa548 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:50.350330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b560543-53c8-4d76-abc7-286b3dffa548 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:50.350599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b560543-53c8-4d76-abc7-286b3dffa548 None admin] PATH_INFO: `/v3/projects/9d41aad68c264e76adc4449c056a869b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:50.351726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b560543-53c8-4d76-abc7-286b3dffa548 None admin] RBAC: Authorizing `identity:get_project(project_id=9d41aad68c264e76adc4449c056a869b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:50.353150 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b560543-53c8-4d76-abc7-286b3dffa548 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:50.354776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 544/1091] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:49:50 2022] GET /identity/v3/projects/9d41aad68c264e76adc4449c056a869b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:50.360856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2bcc-9a01-42bb-84ab-11e36b978b49 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:50.456680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2bcc-9a01-42bb-84ab-11e36b978b49 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:50.490742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfa216-638a-42cb-a8c2-56abe3f4c6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:50.491381 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfa216-638a-42cb-a8c2-56abe3f4c6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:50.491663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfa216-638a-42cb-a8c2-56abe3f4c6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:50.491930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfa216-638a-42cb-a8c2-56abe3f4c6d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:50.492645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfa216-638a-42cb-a8c2-56abe3f4c6d1 None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=9d41aad68c264e76adc4449c056a869b, user.id=890c1a918b88488fb67e6ad7d3ec816a) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:50.492949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfa216-638a-42cb-a8c2-56abe3f4c6d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:50.494367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfa216-638a-42cb-a8c2-56abe3f4c6d1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:50.500159 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 548/1092] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:50 2022] GET /identity/v3/role_assignments?user.id=890c1a918b88488fb67e6ad7d3ec816a&scope.project.id=9d41aad68c264e76adc4449c056a869b&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:50.605883 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eba766-1a1e-45de-bc80-5afe647eb835 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:50.606069 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eba766-1a1e-45de-bc80-5afe647eb835 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:50.606343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eba766-1a1e-45de-bc80-5afe647eb835 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:50.607763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 545/1093] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:50.613394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2080a7e-9778-40e3-b17d-8a62a3291263 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:50.613692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2080a7e-9778-40e3-b17d-8a62a3291263 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:50.613970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2080a7e-9778-40e3-b17d-8a62a3291263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:50.632871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d2080a7e-9778-40e3-b17d-8a62a3291263 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:50.634085 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d2080a7e-9778-40e3-b17d-8a62a3291263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:50.658981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:50.706177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 549/1094] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:50.969649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eba766-1a1e-45de-bc80-5afe647eb835 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:50.973477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:51.069684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eba766-1a1e-45de-bc80-5afe647eb835 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:51.103111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf6728-fdc1-4bd8-8674-64f7cc912936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:51.103726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf6728-fdc1-4bd8-8674-64f7cc912936 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:51.104000 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf6728-fdc1-4bd8-8674-64f7cc912936 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:51.104268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf6728-fdc1-4bd8-8674-64f7cc912936 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:51.114374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf6728-fdc1-4bd8-8674-64f7cc912936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-69bf6728-fdc1-4bd8-8674-64f7cc912936 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:49:51.119097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:51.120195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:49:51.121705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:51.123144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 546/1095] 10.222.0.86 () {56 vars in 1120 bytes} [Mon Aug 29 18:49:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:51.127121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2080a7e-9778-40e3-b17d-8a62a3291263 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:51.224221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2080a7e-9778-40e3-b17d-8a62a3291263 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:51.257469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01c0c0-23eb-44db-ac3d-aad808cb35cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:51.258070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c0c0-23eb-44db-ac3d-aad808cb35cd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:51.258347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c0c0-23eb-44db-ac3d-aad808cb35cd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:51.258616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c0c0-23eb-44db-ac3d-aad808cb35cd None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:51.259367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01c0c0-23eb-44db-ac3d-aad808cb35cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:51.259670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01c0c0-23eb-44db-ac3d-aad808cb35cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:51.261024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01c0c0-23eb-44db-ac3d-aad808cb35cd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:51.268680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 550/1096] 10.222.0.86 () {56 vars in 1115 bytes} [Mon Aug 29 18:49:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:51.274402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf6728-fdc1-4bd8-8674-64f7cc912936 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:51.335627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:51.372824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf6728-fdc1-4bd8-8674-64f7cc912936 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:51.406587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23feb13-5131-4ed8-9e45-2713a8e82c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:51.407241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23feb13-5131-4ed8-9e45-2713a8e82c14 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:51.407517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23feb13-5131-4ed8-9e45-2713a8e82c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:51.407794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23feb13-5131-4ed8-9e45-2713a8e82c14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:51.417541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23feb13-5131-4ed8-9e45-2713a8e82c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:51.418972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23feb13-5131-4ed8-9e45-2713a8e82c14 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e23feb13-5131-4ed8-9e45-2713a8e82c14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:51.432973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:51.433932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:51.434978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:51.434978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:51.434978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:51.434978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 547/1097] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:51.439925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01c0c0-23eb-44db-ac3d-aad808cb35cd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:51.462692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:51.542272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01c0c0-23eb-44db-ac3d-aad808cb35cd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:51.575295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86581fa-e389-49c8-a4f5-81f3f9b85562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:51.575898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86581fa-e389-49c8-a4f5-81f3f9b85562 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:51.576178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86581fa-e389-49c8-a4f5-81f3f9b85562 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:51.576448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86581fa-e389-49c8-a4f5-81f3f9b85562 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:51.577079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86581fa-e389-49c8-a4f5-81f3f9b85562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:51.577379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86581fa-e389-49c8-a4f5-81f3f9b85562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:51.578733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86581fa-e389-49c8-a4f5-81f3f9b85562 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:51.591956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 551/1098] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:51.599873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23feb13-5131-4ed8-9e45-2713a8e82c14 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:51.698723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23feb13-5131-4ed8-9e45-2713a8e82c14 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:51.714806 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:51.736213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5452f-2444-4d93-8087-abcf7cbcf342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:51.736845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5452f-2444-4d93-8087-abcf7cbcf342 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:51.737128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5452f-2444-4d93-8087-abcf7cbcf342 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:51.737402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5452f-2444-4d93-8087-abcf7cbcf342 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:51.745304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5452f-2444-4d93-8087-abcf7cbcf342 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:51.746817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5452f-2444-4d93-8087-abcf7cbcf342 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-32b5452f-2444-4d93-8087-abcf7cbcf342 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:49:51.757899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:51.763932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:51.765121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:51.765121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:51.765121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:49:51.765121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:49:51.765121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:49:51.765121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:49:51.765121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:49:51.765121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:49:51.765121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:49:51.765121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:49:51.765121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 548/1099] 10.222.0.86 () {56 vars in 1093 bytes} [Mon Aug 29 18:49:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:49:51.765726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86581fa-e389-49c8-a4f5-81f3f9b85562 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:51.846165 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:51.865805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86581fa-e389-49c8-a4f5-81f3f9b85562 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:51.899684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7a41e-042f-494c-9c24-7f2f4bb8b7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:51.900366 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7a41e-042f-494c-9c24-7f2f4bb8b7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:51.900674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7a41e-042f-494c-9c24-7f2f4bb8b7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:51.900917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7a41e-042f-494c-9c24-7f2f4bb8b7e9 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:51.901623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7a41e-042f-494c-9c24-7f2f4bb8b7e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:51.902463 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7a41e-042f-494c-9c24-7f2f4bb8b7e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:51.903895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7a41e-042f-494c-9c24-7f2f4bb8b7e9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:51.920948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 552/1100] 10.222.0.86 () {56 vars in 1133 bytes} [Mon Aug 29 18:49:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:51.926458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5452f-2444-4d93-8087-abcf7cbcf342 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:52.024062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5452f-2444-4d93-8087-abcf7cbcf342 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:52.060968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbba0e-6e02-4ca1-a3cb-77e6e63293ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:52.061827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbba0e-6e02-4ca1-a3cb-77e6e63293ef None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:52.062234 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbba0e-6e02-4ca1-a3cb-77e6e63293ef None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:52.062629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbba0e-6e02-4ca1-a3cb-77e6e63293ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:52.117822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:52.141625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbba0e-6e02-4ca1-a3cb-77e6e63293ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:52.143099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbba0e-6e02-4ca1-a3cb-77e6e63293ef None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:52.238800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 549/1101] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:49:52.244578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7a41e-042f-494c-9c24-7f2f4bb8b7e9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:52.342631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7a41e-042f-494c-9c24-7f2f4bb8b7e9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:52.375403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe404514-3ab1-4a0f-9fc9-02fec5add69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:52.375948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe404514-3ab1-4a0f-9fc9-02fec5add69e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:52.376219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe404514-3ab1-4a0f-9fc9-02fec5add69e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:52.376484 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe404514-3ab1-4a0f-9fc9-02fec5add69e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:52.382837 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:52.387941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe404514-3ab1-4a0f-9fc9-02fec5add69e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:52.389798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe404514-3ab1-4a0f-9fc9-02fec5add69e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fe404514-3ab1-4a0f-9fc9-02fec5add69e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:49:52.408123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:52.409182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:49:52.410095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:49:52.410095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:49:52.410095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:52.410095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 553/1102] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:52.415026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbba0e-6e02-4ca1-a3cb-77e6e63293ef None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:52.441806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359d5d-b31d-4825-a679-06dc8e941e5d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:52.442100 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359d5d-b31d-4825-a679-06dc8e941e5d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:52.442373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359d5d-b31d-4825-a679-06dc8e941e5d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:52.443735 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 554/1103] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:49:52.448732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b203132-5348-4bd9-8d2c-01663dd77a95 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:52.449509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b203132-5348-4bd9-8d2c-01663dd77a95 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:52.449782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b203132-5348-4bd9-8d2c-01663dd77a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:52.468007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1b203132-5348-4bd9-8d2c-01663dd77a95 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:52.469183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1b203132-5348-4bd9-8d2c-01663dd77a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:52.489778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:52.513304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbba0e-6e02-4ca1-a3cb-77e6e63293ef None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:52.535740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 555/1104] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:49:52.546117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63c3a4-7c93-44dd-a4e6-728a9421a27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:52.546783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63c3a4-7c93-44dd-a4e6-728a9421a27c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:52.547105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63c3a4-7c93-44dd-a4e6-728a9421a27c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:52.547384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63c3a4-7c93-44dd-a4e6-728a9421a27c None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:52.548057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63c3a4-7c93-44dd-a4e6-728a9421a27c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:52.548359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63c3a4-7c93-44dd-a4e6-728a9421a27c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:52.549710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63c3a4-7c93-44dd-a4e6-728a9421a27c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:52.561574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 550/1105] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:52.567244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b203132-5348-4bd9-8d2c-01663dd77a95 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:52.666921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b203132-5348-4bd9-8d2c-01663dd77a95 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:52.689081 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:52.703415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80cae9-ea1e-4d28-9ddb-933da5c36592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:52.704047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80cae9-ea1e-4d28-9ddb-933da5c36592 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:52.704324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80cae9-ea1e-4d28-9ddb-933da5c36592 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:52.704594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80cae9-ea1e-4d28-9ddb-933da5c36592 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:52.711130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80cae9-ea1e-4d28-9ddb-933da5c36592 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:52.712554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80cae9-ea1e-4d28-9ddb-933da5c36592 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7c80cae9-ea1e-4d28-9ddb-933da5c36592 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:49:52.723436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:49:52.724335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:49:52.725576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:49:52.725576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:49:52.725576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:49:52.725576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:49:52.725576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:49:52.725576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:49:52.725576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:49:52.725576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:49:52.725576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 556/1106] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:49:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:49:52.730819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63c3a4-7c93-44dd-a4e6-728a9421a27c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:52.802787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80cae9-ea1e-4d28-9ddb-933da5c36592 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:52.828433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63c3a4-7c93-44dd-a4e6-728a9421a27c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:52.861631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c133955-00b7-42d1-bc25-4b8fb5368a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:52.862228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c133955-00b7-42d1-bc25-4b8fb5368a58 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:52.862507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c133955-00b7-42d1-bc25-4b8fb5368a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:52.862786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c133955-00b7-42d1-bc25-4b8fb5368a58 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:52.863499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c133955-00b7-42d1-bc25-4b8fb5368a58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:52.863803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c133955-00b7-42d1-bc25-4b8fb5368a58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:52.865203 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c133955-00b7-42d1-bc25-4b8fb5368a58 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:52.867672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:52.878110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 551/1107] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:49:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:52.884186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c133955-00b7-42d1-bc25-4b8fb5368a58 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:52.906922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80cae9-ea1e-4d28-9ddb-933da5c36592 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:52.941038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19deea-a07d-4996-8233-07452f03d163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:52.941597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19deea-a07d-4996-8233-07452f03d163 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:52.941883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19deea-a07d-4996-8233-07452f03d163 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:52.942153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19deea-a07d-4996-8233-07452f03d163 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:52.943287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19deea-a07d-4996-8233-07452f03d163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:52.945024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19deea-a07d-4996-8233-07452f03d163 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:52.946543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 557/1108] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:49:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:52.952731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19deea-a07d-4996-8233-07452f03d163 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:52.983301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c133955-00b7-42d1-bc25-4b8fb5368a58 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:53.001502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:53.017505 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915989dd-5b40-43cd-96a0-c1065e5e8ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:53.018109 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915989dd-5b40-43cd-96a0-c1065e5e8ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:53.018389 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915989dd-5b40-43cd-96a0-c1065e5e8ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:53.018676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915989dd-5b40-43cd-96a0-c1065e5e8ee0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:53.019402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915989dd-5b40-43cd-96a0-c1065e5e8ee0 None admin] RBAC: Adding query filter params (role.id=4ee6c8fb1f294eb0824448e1f26984d6, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=045b22d046c549f2b53d2ed73b94f936) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:53.019710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915989dd-5b40-43cd-96a0-c1065e5e8ee0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:53.021132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915989dd-5b40-43cd-96a0-c1065e5e8ee0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:53.026884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 552/1109] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:52 2022] GET /identity/v3/role_assignments?user.id=045b22d046c549f2b53d2ed73b94f936&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=4ee6c8fb1f294eb0824448e1f26984d6 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:53.054284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19deea-a07d-4996-8233-07452f03d163 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:53.091454 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf8eaf-aba1-4cdc-8c9c-3378c27cb95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:53.092077 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf8eaf-aba1-4cdc-8c9c-3378c27cb95b None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:53.092351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf8eaf-aba1-4cdc-8c9c-3378c27cb95b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:53.092622 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf8eaf-aba1-4cdc-8c9c-3378c27cb95b None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:53.093276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf8eaf-aba1-4cdc-8c9c-3378c27cb95b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:53.094737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf8eaf-aba1-4cdc-8c9c-3378c27cb95b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:53.117905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 558/1110] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:49:52 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:49:55.064240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de99474-78bd-4bdb-9771-3d7fe83c49e8 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:55.064794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de99474-78bd-4bdb-9771-3d7fe83c49e8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:55.064794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de99474-78bd-4bdb-9771-3d7fe83c49e8 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:55.066136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 553/1111] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:55.071530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac85d1-c293-426d-82a5-227abb9b03f3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:55.071899 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac85d1-c293-426d-82a5-227abb9b03f3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:55.072078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac85d1-c293-426d-82a5-227abb9b03f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:55.090983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-71ac85d1-c293-426d-82a5-227abb9b03f3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:55.092185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-71ac85d1-c293-426d-82a5-227abb9b03f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:55.156370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 559/1112] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:55.423229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de99474-78bd-4bdb-9771-3d7fe83c49e8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:55.482923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:55.524303 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de99474-78bd-4bdb-9771-3d7fe83c49e8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:55.557120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d969ab-144c-42b0-9a63-53b7ad6c2588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:55.557731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d969ab-144c-42b0-9a63-53b7ad6c2588 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:55.558530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d969ab-144c-42b0-9a63-53b7ad6c2588 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:55.558798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d969ab-144c-42b0-9a63-53b7ad6c2588 None admin] PATH_INFO: `/v3/roles/31cc9f3a35594d40924277f4778c9f03` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:55.560197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d969ab-144c-42b0-9a63-53b7ad6c2588 None admin] RBAC: Authorizing `identity:get_role(role_id=31cc9f3a35594d40924277f4778c9f03)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:55.561672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d969ab-144c-42b0-9a63-53b7ad6c2588 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:55.562944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 554/1113] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:49:55 2022] GET /identity/v3/roles/31cc9f3a35594d40924277f4778c9f03 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:55.568726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac85d1-c293-426d-82a5-227abb9b03f3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:55.602181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:55.669926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac85d1-c293-426d-82a5-227abb9b03f3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:55.703632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9966b-8b51-4479-9849-43c059e5e5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:55.704260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9966b-8b51-4479-9849-43c059e5e5cd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:55.704542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9966b-8b51-4479-9849-43c059e5e5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:55.704810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9966b-8b51-4479-9849-43c059e5e5cd None admin] PATH_INFO: `/v3/users/128154283c764715bcd63cc9e3ea9993` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:55.717379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9966b-8b51-4479-9849-43c059e5e5cd None admin] RBAC: Authorizing `identity:get_user(user_id=128154283c764715bcd63cc9e3ea9993)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:55.718810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9966b-8b51-4479-9849-43c059e5e5cd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:55.720437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 560/1114] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:49:55 2022] GET /identity/v3/users/128154283c764715bcd63cc9e3ea9993 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:55.726231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d969ab-144c-42b0-9a63-53b7ad6c2588 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:55.828821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d969ab-144c-42b0-9a63-53b7ad6c2588 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:55.854123 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:55.863186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d9eae-9e4f-4ee2-8868-893a2caa3998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:55.863844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9eae-9e4f-4ee2-8868-893a2caa3998 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:55.864129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9eae-9e4f-4ee2-8868-893a2caa3998 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:55.864401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9eae-9e4f-4ee2-8868-893a2caa3998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:55.928881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d9eae-9e4f-4ee2-8868-893a2caa3998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:55.930315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d9eae-9e4f-4ee2-8868-893a2caa3998 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:56.026636 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 555/1115] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:49:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:49:56.033819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9966b-8b51-4479-9849-43c059e5e5cd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:56.128414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:56.137187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9966b-8b51-4479-9849-43c059e5e5cd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:56.172717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9d060-99d8-446a-be0c-16fa23fb4f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:56.173344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9d060-99d8-446a-be0c-16fa23fb4f2d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:56.173625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9d060-99d8-446a-be0c-16fa23fb4f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:56.173900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9d060-99d8-446a-be0c-16fa23fb4f2d None admin] PATH_INFO: `/v3/projects/9d41aad68c264e76adc4449c056a869b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:56.175023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9d060-99d8-446a-be0c-16fa23fb4f2d None admin] RBAC: Authorizing `identity:get_project(project_id=9d41aad68c264e76adc4449c056a869b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:56.176472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9d060-99d8-446a-be0c-16fa23fb4f2d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:56.178093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 561/1116] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:49:56 2022] GET /identity/v3/projects/9d41aad68c264e76adc4449c056a869b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:56.184375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d9eae-9e4f-4ee2-8868-893a2caa3998 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:56.211292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:56.282046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d9eae-9e4f-4ee2-8868-893a2caa3998 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:56.314730 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032ec5b-cff6-49a9-8649-b8d330a8caa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:56.315420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032ec5b-cff6-49a9-8649-b8d330a8caa0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:56.315698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032ec5b-cff6-49a9-8649-b8d330a8caa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:56.315966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032ec5b-cff6-49a9-8649-b8d330a8caa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:56.316671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032ec5b-cff6-49a9-8649-b8d330a8caa0 None admin] RBAC: Adding query filter params (role.id=31cc9f3a35594d40924277f4778c9f03, scope.project.id=9d41aad68c264e76adc4449c056a869b, user.id=128154283c764715bcd63cc9e3ea9993) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:49:56.316979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032ec5b-cff6-49a9-8649-b8d330a8caa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:56.318376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032ec5b-cff6-49a9-8649-b8d330a8caa0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:56.324274 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 556/1117] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:49:56 2022] GET /identity/v3/role_assignments?user.id=128154283c764715bcd63cc9e3ea9993&scope.project.id=9d41aad68c264e76adc4449c056a869b&role.id=31cc9f3a35594d40924277f4778c9f03 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:58.228019 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f572df-ae4b-462a-893c-3e1437d7b376 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:58.228519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f572df-ae4b-462a-893c-3e1437d7b376 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:58.228611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f572df-ae4b-462a-893c-3e1437d7b376 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:58.229933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 562/1118] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:49:58.235130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9afc8-757e-4010-b8a6-71e33cefaac1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:58.235530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9afc8-757e-4010-b8a6-71e33cefaac1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:58.235680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9afc8-757e-4010-b8a6-71e33cefaac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:58.253936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bca9afc8-757e-4010-b8a6-71e33cefaac1 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:49:58.255143 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bca9afc8-757e-4010-b8a6-71e33cefaac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:49:58.319059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 557/1119] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:49:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:49:58.642218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f572df-ae4b-462a-893c-3e1437d7b376 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:58.741966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f572df-ae4b-462a-893c-3e1437d7b376 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:58.774564 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389338e-60f0-43c6-b235-43f462c1389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:58.775210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389338e-60f0-43c6-b235-43f462c1389e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:58.775490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389338e-60f0-43c6-b235-43f462c1389e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:58.775764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389338e-60f0-43c6-b235-43f462c1389e None admin] PATH_INFO: `/v3/roles/31cc9f3a35594d40924277f4778c9f03` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:58.777167 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389338e-60f0-43c6-b235-43f462c1389e None admin] RBAC: Authorizing `identity:get_role(role_id=31cc9f3a35594d40924277f4778c9f03)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:58.778521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389338e-60f0-43c6-b235-43f462c1389e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:58.779848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 563/1120] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:49:58 2022] GET /identity/v3/roles/31cc9f3a35594d40924277f4778c9f03 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:58.785723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9afc8-757e-4010-b8a6-71e33cefaac1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:58.809606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:58.883549 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9afc8-757e-4010-b8a6-71e33cefaac1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:58.924881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15900d8-7d4b-46f7-98b5-3f76015aeef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:58.925430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15900d8-7d4b-46f7-98b5-3f76015aeef6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:58.925711 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15900d8-7d4b-46f7-98b5-3f76015aeef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:58.925982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15900d8-7d4b-46f7-98b5-3f76015aeef6 None admin] PATH_INFO: `/v3/users/128154283c764715bcd63cc9e3ea9993` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:58.927322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15900d8-7d4b-46f7-98b5-3f76015aeef6 None admin] RBAC: Authorizing `identity:get_user(user_id=128154283c764715bcd63cc9e3ea9993)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:58.930079 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15900d8-7d4b-46f7-98b5-3f76015aeef6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:58.931677 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 558/1121] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:49:58 2022] GET /identity/v3/users/128154283c764715bcd63cc9e3ea9993 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:49:58.938027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389338e-60f0-43c6-b235-43f462c1389e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:58.967757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:59.037350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389338e-60f0-43c6-b235-43f462c1389e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:59.073142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b71cf-f8de-481d-9a60-9e8f2ab4a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:59.073790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b71cf-f8de-481d-9a60-9e8f2ab4a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:59.074071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b71cf-f8de-481d-9a60-9e8f2ab4a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:59.074343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b71cf-f8de-481d-9a60-9e8f2ab4a3b8 None admin] PATH_INFO: `/v3/projects/9d41aad68c264e76adc4449c056a869b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:59.075472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b71cf-f8de-481d-9a60-9e8f2ab4a3b8 None admin] RBAC: Authorizing `identity:get_project(project_id=9d41aad68c264e76adc4449c056a869b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:59.076894 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b71cf-f8de-481d-9a60-9e8f2ab4a3b8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:59.078509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 564/1122] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:49:58 2022] GET /identity/v3/projects/9d41aad68c264e76adc4449c056a869b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:49:59.084428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15900d8-7d4b-46f7-98b5-3f76015aeef6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:49:59.172791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:49:59.183900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15900d8-7d4b-46f7-98b5-3f76015aeef6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:49:59.217478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63006b-ab65-43f0-be02-d609de36efa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:49:59.218120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63006b-ab65-43f0-be02-d609de36efa1 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:49:59.218397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63006b-ab65-43f0-be02-d609de36efa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:49:59.218667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63006b-ab65-43f0-be02-d609de36efa1 None admin] PATH_INFO: `/v3/projects/9d41aad68c264e76adc4449c056a869b/users/128154283c764715bcd63cc9e3ea9993/roles/31cc9f3a35594d40924277f4778c9f03` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:49:59.220800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63006b-ab65-43f0-be02-d609de36efa1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9d41aad68c264e76adc4449c056a869b, user_id=128154283c764715bcd63cc9e3ea9993, role_id=31cc9f3a35594d40924277f4778c9f03)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:49:59.222209 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63006b-ab65-43f0-be02-d609de36efa1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:49:59.235388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 559/1123] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:49:59 2022] PUT /identity/v3/projects/9d41aad68c264e76adc4449c056a869b/users/128154283c764715bcd63cc9e3ea9993/roles/31cc9f3a35594d40924277f4778c9f03 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:50:01.134538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45d6ef-bae3-4584-a4d1-8e0fbbda82ec None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:01.135113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45d6ef-bae3-4584-a4d1-8e0fbbda82ec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:01.135113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45d6ef-bae3-4584-a4d1-8e0fbbda82ec None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:01.137325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 565/1124] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:01.142481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b9167-d9fe-4146-b563-b4912efa359f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:01.142898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b9167-d9fe-4146-b563-b4912efa359f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:01.143057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b9167-d9fe-4146-b563-b4912efa359f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:01.161879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-323b9167-d9fe-4146-b563-b4912efa359f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:01.163149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-323b9167-d9fe-4146-b563-b4912efa359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:01.234862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 560/1125] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:01.502530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45d6ef-bae3-4584-a4d1-8e0fbbda82ec None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:01.595207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:01.601945 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45d6ef-bae3-4584-a4d1-8e0fbbda82ec None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:01.634989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551225f-f361-4c78-8b7b-4b4eed77d46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:01.635623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551225f-f361-4c78-8b7b-4b4eed77d46e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:01.635912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551225f-f361-4c78-8b7b-4b4eed77d46e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:01.636191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551225f-f361-4c78-8b7b-4b4eed77d46e None admin] PATH_INFO: `/v3/roles/31cc9f3a35594d40924277f4778c9f03` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:01.637553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551225f-f361-4c78-8b7b-4b4eed77d46e None admin] RBAC: Authorizing `identity:get_role(role_id=31cc9f3a35594d40924277f4778c9f03)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:01.638914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551225f-f361-4c78-8b7b-4b4eed77d46e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:01.640234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 566/1126] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:50:01 2022] GET /identity/v3/roles/31cc9f3a35594d40924277f4778c9f03 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:01.645954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b9167-d9fe-4146-b563-b4912efa359f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:01.726920 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:01.746937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b9167-d9fe-4146-b563-b4912efa359f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:01.778788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c385f57-5f0d-4553-a273-8d3f8242fd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:01.779430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c385f57-5f0d-4553-a273-8d3f8242fd9a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:01.779704 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c385f57-5f0d-4553-a273-8d3f8242fd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:01.779978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c385f57-5f0d-4553-a273-8d3f8242fd9a None admin] PATH_INFO: `/v3/users/128154283c764715bcd63cc9e3ea9993` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:01.781073 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c385f57-5f0d-4553-a273-8d3f8242fd9a None admin] RBAC: Authorizing `identity:get_user(user_id=128154283c764715bcd63cc9e3ea9993)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:01.782481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c385f57-5f0d-4553-a273-8d3f8242fd9a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:01.784054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 561/1127] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:50:01 2022] GET /identity/v3/users/128154283c764715bcd63cc9e3ea9993 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:01.789781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551225f-f361-4c78-8b7b-4b4eed77d46e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:01.886771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551225f-f361-4c78-8b7b-4b4eed77d46e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:01.919318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be126fe-223f-4abc-a2a3-7f9e64b26334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:01.919912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be126fe-223f-4abc-a2a3-7f9e64b26334 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:01.920186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be126fe-223f-4abc-a2a3-7f9e64b26334 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:01.920455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be126fe-223f-4abc-a2a3-7f9e64b26334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:01.949966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:01.986318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be126fe-223f-4abc-a2a3-7f9e64b26334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:01.987773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be126fe-223f-4abc-a2a3-7f9e64b26334 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:02.081144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 567/1128] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:50:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:50:02.086968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c385f57-5f0d-4553-a273-8d3f8242fd9a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:02.183602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c385f57-5f0d-4553-a273-8d3f8242fd9a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:02.216177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3b5e4-1778-4b0f-aa96-e5dd1308c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:02.216787 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3b5e4-1778-4b0f-aa96-e5dd1308c479 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:02.217063 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3b5e4-1778-4b0f-aa96-e5dd1308c479 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:02.217331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3b5e4-1778-4b0f-aa96-e5dd1308c479 None admin] PATH_INFO: `/v3/projects/9d41aad68c264e76adc4449c056a869b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:02.218416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3b5e4-1778-4b0f-aa96-e5dd1308c479 None admin] RBAC: Authorizing `identity:get_project(project_id=9d41aad68c264e76adc4449c056a869b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:02.219852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3b5e4-1778-4b0f-aa96-e5dd1308c479 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:02.221439 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 562/1129] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:50:02 2022] GET /identity/v3/projects/9d41aad68c264e76adc4449c056a869b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:02.227771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be126fe-223f-4abc-a2a3-7f9e64b26334 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:02.287367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:02.325420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be126fe-223f-4abc-a2a3-7f9e64b26334 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:02.358300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6d0b4-5cd6-4391-a168-f1768dc9045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:02.358834 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d0b4-5cd6-4391-a168-f1768dc9045e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:02.359143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d0b4-5cd6-4391-a168-f1768dc9045e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:02.359415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d0b4-5cd6-4391-a168-f1768dc9045e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:02.360124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6d0b4-5cd6-4391-a168-f1768dc9045e None admin] RBAC: Adding query filter params (role.id=31cc9f3a35594d40924277f4778c9f03, scope.project.id=9d41aad68c264e76adc4449c056a869b, user.id=128154283c764715bcd63cc9e3ea9993) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:02.360429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6d0b4-5cd6-4391-a168-f1768dc9045e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:02.361840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6d0b4-5cd6-4391-a168-f1768dc9045e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:02.367419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 568/1130] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:50:02 2022] GET /identity/v3/role_assignments?user.id=128154283c764715bcd63cc9e3ea9993&scope.project.id=9d41aad68c264e76adc4449c056a869b&role.id=31cc9f3a35594d40924277f4778c9f03 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:04.267572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d102f-533a-491a-963a-c3cd1995eb3c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:04.268144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d102f-533a-491a-963a-c3cd1995eb3c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:04.268144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d102f-533a-491a-963a-c3cd1995eb3c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:04.269431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 563/1131] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:04.274490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c20e2-9b52-4e3a-b224-fda11697fe62 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:04.274907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c20e2-9b52-4e3a-b224-fda11697fe62 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:04.275057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c20e2-9b52-4e3a-b224-fda11697fe62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:04.292822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a40c20e2-9b52-4e3a-b224-fda11697fe62 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:04.293982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a40c20e2-9b52-4e3a-b224-fda11697fe62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:04.358119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 569/1132] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:04.627887 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d102f-533a-491a-963a-c3cd1995eb3c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:04.645681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:04.726660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d102f-533a-491a-963a-c3cd1995eb3c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:04.760235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a8326-c610-48ce-a7b6-0033afaeae77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:04.760873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a8326-c610-48ce-a7b6-0033afaeae77 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:04.761151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a8326-c610-48ce-a7b6-0033afaeae77 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:04.761420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a8326-c610-48ce-a7b6-0033afaeae77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:04.762507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a8326-c610-48ce-a7b6-0033afaeae77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:04.763943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a8326-c610-48ce-a7b6-0033afaeae77 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:04.765476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 564/1133] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:50:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:04.771439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c20e2-9b52-4e3a-b224-fda11697fe62 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:04.829218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:04.869543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c20e2-9b52-4e3a-b224-fda11697fe62 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:04.902457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748229a-760c-490b-a259-2e11df1736bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:04.903015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748229a-760c-490b-a259-2e11df1736bb None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:04.903299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748229a-760c-490b-a259-2e11df1736bb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:04.903573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748229a-760c-490b-a259-2e11df1736bb None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:04.904225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748229a-760c-490b-a259-2e11df1736bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:04.905649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748229a-760c-490b-a259-2e11df1736bb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:04.931725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 570/1134] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:50:04 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:50:06.861819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bacc75-359f-46d6-8b0d-6ee9e4d1f4a5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:06.862375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bacc75-359f-46d6-8b0d-6ee9e4d1f4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:06.862375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bacc75-359f-46d6-8b0d-6ee9e4d1f4a5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:06.863709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 565/1135] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:06.868873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66270477-a2af-4a4c-b1cf-d0de052d7430 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:06.869302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66270477-a2af-4a4c-b1cf-d0de052d7430 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:06.869402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66270477-a2af-4a4c-b1cf-d0de052d7430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:06.887934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-66270477-a2af-4a4c-b1cf-d0de052d7430 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:06.889095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-66270477-a2af-4a4c-b1cf-d0de052d7430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:06.952714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 571/1136] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:07.221927 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bacc75-359f-46d6-8b0d-6ee9e4d1f4a5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:07.303471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:07.320681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bacc75-359f-46d6-8b0d-6ee9e4d1f4a5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:07.353804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3bb86-dff6-4822-87cb-88d15478362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:07.354441 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3bb86-dff6-4822-87cb-88d15478362d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:07.354700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3bb86-dff6-4822-87cb-88d15478362d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:07.354969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3bb86-dff6-4822-87cb-88d15478362d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:07.356090 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3bb86-dff6-4822-87cb-88d15478362d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:07.357657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3bb86-dff6-4822-87cb-88d15478362d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:07.359232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 566/1137] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:50:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:07.364890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66270477-a2af-4a4c-b1cf-d0de052d7430 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:07.404573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:07.472249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66270477-a2af-4a4c-b1cf-d0de052d7430 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:07.513824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d49a7b-a108-4aa3-857b-14db66e6a96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:07.514530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49a7b-a108-4aa3-857b-14db66e6a96f None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:07.514971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49a7b-a108-4aa3-857b-14db66e6a96f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:07.515392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49a7b-a108-4aa3-857b-14db66e6a96f None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:07.516329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d49a7b-a108-4aa3-857b-14db66e6a96f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:07.517742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d49a7b-a108-4aa3-857b-14db66e6a96f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:07.537897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 572/1138] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:50:07 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:50:09.455626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f860a07-43db-4e9b-8d96-6576ff203d76 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:09.456196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f860a07-43db-4e9b-8d96-6576ff203d76 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:09.456196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f860a07-43db-4e9b-8d96-6576ff203d76 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:09.457480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 567/1139] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:09.462650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90115650-1fdc-435b-92c9-78fd3063f54d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:09.463087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90115650-1fdc-435b-92c9-78fd3063f54d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:09.463222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90115650-1fdc-435b-92c9-78fd3063f54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:09.481577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-90115650-1fdc-435b-92c9-78fd3063f54d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:09.482741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-90115650-1fdc-435b-92c9-78fd3063f54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:09.546603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 573/1140] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:09.815011 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f860a07-43db-4e9b-8d96-6576ff203d76 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:09.915794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f860a07-43db-4e9b-8d96-6576ff203d76 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:09.948933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbeb0ec-0ac8-4db1-aa28-21ecc92e34e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:09.949547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbeb0ec-0ac8-4db1-aa28-21ecc92e34e7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:09.949824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbeb0ec-0ac8-4db1-aa28-21ecc92e34e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:09.950092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbeb0ec-0ac8-4db1-aa28-21ecc92e34e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:09.960445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbeb0ec-0ac8-4db1-aa28-21ecc92e34e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-efbeb0ec-0ac8-4db1-aa28-21ecc92e34e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:50:09.965838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:50:09.967016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:50:09.968248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 568/1141] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:50:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:50:09.972905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90115650-1fdc-435b-92c9-78fd3063f54d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:09.998319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:10.068807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90115650-1fdc-435b-92c9-78fd3063f54d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:10.101792 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2accaed-d789-411a-991b-61616d984dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:10.102384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2accaed-d789-411a-991b-61616d984dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:10.102661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2accaed-d789-411a-991b-61616d984dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:10.102930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2accaed-d789-411a-991b-61616d984dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:10.103612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2accaed-d789-411a-991b-61616d984dc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:10.103909 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2accaed-d789-411a-991b-61616d984dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:10.105284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2accaed-d789-411a-991b-61616d984dc6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:10.112953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 574/1142] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:50:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:10.118640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbeb0ec-0ac8-4db1-aa28-21ecc92e34e7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:10.133472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:10.215678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbeb0ec-0ac8-4db1-aa28-21ecc92e34e7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:10.248057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f6942-3708-4b8a-970d-9e82c634d237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:10.248668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f6942-3708-4b8a-970d-9e82c634d237 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:10.248946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f6942-3708-4b8a-970d-9e82c634d237 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:10.249213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f6942-3708-4b8a-970d-9e82c634d237 None admin] PATH_INFO: `/v3/users/46216a7e489a4ec8bdd65975480d698c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:10.262434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f6942-3708-4b8a-970d-9e82c634d237 None admin] RBAC: Authorizing `identity:get_user(user_id=46216a7e489a4ec8bdd65975480d698c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:10.263933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f6942-3708-4b8a-970d-9e82c634d237 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:10.265566 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 569/1143] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:50:10 2022] GET /identity/v3/users/46216a7e489a4ec8bdd65975480d698c => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:10.271493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2accaed-d789-411a-991b-61616d984dc6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:10.357281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:10.369193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2accaed-d789-411a-991b-61616d984dc6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:10.402172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746642b-f8ab-49ba-8050-a741f52f669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:10.402769 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746642b-f8ab-49ba-8050-a741f52f669e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:10.403084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746642b-f8ab-49ba-8050-a741f52f669e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:10.403364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746642b-f8ab-49ba-8050-a741f52f669e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:10.468546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746642b-f8ab-49ba-8050-a741f52f669e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:10.469989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746642b-f8ab-49ba-8050-a741f52f669e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:10.551891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:10.565090 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 575/1144] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:50:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:50:10.571259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f6942-3708-4b8a-970d-9e82c634d237 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:10.639136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:10.669656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f6942-3708-4b8a-970d-9e82c634d237 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:10.704178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7856d-59ca-478b-bd10-3f6df39d6645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:10.704798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7856d-59ca-478b-bd10-3f6df39d6645 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:10.705074 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7856d-59ca-478b-bd10-3f6df39d6645 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:10.705343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7856d-59ca-478b-bd10-3f6df39d6645 None admin] PATH_INFO: `/v3/projects/b62fce02f29347a1b707951734566cc6` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:10.706420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7856d-59ca-478b-bd10-3f6df39d6645 None admin] RBAC: Authorizing `identity:get_project(project_id=b62fce02f29347a1b707951734566cc6)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:10.707857 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7856d-59ca-478b-bd10-3f6df39d6645 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:10.709454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 570/1145] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:50:10 2022] GET /identity/v3/projects/b62fce02f29347a1b707951734566cc6 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:10.715450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746642b-f8ab-49ba-8050-a741f52f669e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:10.809967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746642b-f8ab-49ba-8050-a741f52f669e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:10.842448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc40e6-8ef3-4e39-8e18-c63e6d57331b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:10.843058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc40e6-8ef3-4e39-8e18-c63e6d57331b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:10.843334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc40e6-8ef3-4e39-8e18-c63e6d57331b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:10.843600 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc40e6-8ef3-4e39-8e18-c63e6d57331b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:10.844302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc40e6-8ef3-4e39-8e18-c63e6d57331b None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=b62fce02f29347a1b707951734566cc6, user.id=46216a7e489a4ec8bdd65975480d698c) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:10.844604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc40e6-8ef3-4e39-8e18-c63e6d57331b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:10.846014 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc40e6-8ef3-4e39-8e18-c63e6d57331b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:10.851190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 576/1146] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:50:10 2022] GET /identity/v3/role_assignments?user.id=46216a7e489a4ec8bdd65975480d698c&scope.project.id=b62fce02f29347a1b707951734566cc6&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:12.720218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7008ee7-b22a-48a0-b2c0-bc89a85eafe7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:12.720691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7008ee7-b22a-48a0-b2c0-bc89a85eafe7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:12.720792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7008ee7-b22a-48a0-b2c0-bc89a85eafe7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:12.722068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 571/1147] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:12.727208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13834-7bcd-431a-bde5-e44ca3c8be80 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:12.727627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13834-7bcd-431a-bde5-e44ca3c8be80 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:12.727754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13834-7bcd-431a-bde5-e44ca3c8be80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:12.745927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4ed13834-7bcd-431a-bde5-e44ca3c8be80 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:12.747127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4ed13834-7bcd-431a-bde5-e44ca3c8be80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:12.791799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:12.811669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 577/1148] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:13.077005 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7008ee7-b22a-48a0-b2c0-bc89a85eafe7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:13.174825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7008ee7-b22a-48a0-b2c0-bc89a85eafe7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:13.208242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96cb5f-d258-44a4-a344-02b8fec24110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:13.208864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96cb5f-d258-44a4-a344-02b8fec24110 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:13.209147 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96cb5f-d258-44a4-a344-02b8fec24110 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:13.209422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96cb5f-d258-44a4-a344-02b8fec24110 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:13.216606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:13.220621 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96cb5f-d258-44a4-a344-02b8fec24110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0c96cb5f-d258-44a4-a344-02b8fec24110 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:50:13.225572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:50:13.226650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:50:13.227817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 572/1149] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:50:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:50:13.233299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed13834-7bcd-431a-bde5-e44ca3c8be80 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:13.328387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed13834-7bcd-431a-bde5-e44ca3c8be80 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:13.360951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e18742-dca3-4fe1-b44a-d2c9c88b7e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:13.361549 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18742-dca3-4fe1-b44a-d2c9c88b7e21 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:13.361828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18742-dca3-4fe1-b44a-d2c9c88b7e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:13.362094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18742-dca3-4fe1-b44a-d2c9c88b7e21 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:13.362739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e18742-dca3-4fe1-b44a-d2c9c88b7e21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:13.363063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e18742-dca3-4fe1-b44a-d2c9c88b7e21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:13.364410 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e18742-dca3-4fe1-b44a-d2c9c88b7e21 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:13.372709 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 578/1150] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:50:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:13.378557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96cb5f-d258-44a4-a344-02b8fec24110 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:13.473932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96cb5f-d258-44a4-a344-02b8fec24110 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:13.508057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e59c1b-083f-4041-82df-50a4a000e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:13.508672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59c1b-083f-4041-82df-50a4a000e2bc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:13.508947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59c1b-083f-4041-82df-50a4a000e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:13.509217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59c1b-083f-4041-82df-50a4a000e2bc None admin] PATH_INFO: `/v3/users/46216a7e489a4ec8bdd65975480d698c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:13.510315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e59c1b-083f-4041-82df-50a4a000e2bc None admin] RBAC: Authorizing `identity:get_user(user_id=46216a7e489a4ec8bdd65975480d698c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:13.512386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e59c1b-083f-4041-82df-50a4a000e2bc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:13.513937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 573/1151] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:50:13 2022] GET /identity/v3/users/46216a7e489a4ec8bdd65975480d698c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:13.519703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e18742-dca3-4fe1-b44a-d2c9c88b7e21 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:13.563829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:13.616548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e18742-dca3-4fe1-b44a-d2c9c88b7e21 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:13.648559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de029e84-7bf2-4183-bd24-0193cbac330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:13.649177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de029e84-7bf2-4183-bd24-0193cbac330c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:13.649455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de029e84-7bf2-4183-bd24-0193cbac330c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:13.649727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de029e84-7bf2-4183-bd24-0193cbac330c None admin] PATH_INFO: `/v3/projects/b62fce02f29347a1b707951734566cc6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:13.650821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de029e84-7bf2-4183-bd24-0193cbac330c None admin] RBAC: Authorizing `identity:get_project(project_id=b62fce02f29347a1b707951734566cc6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:13.652270 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de029e84-7bf2-4183-bd24-0193cbac330c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:13.653876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 579/1152] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:50:13 2022] GET /identity/v3/projects/b62fce02f29347a1b707951734566cc6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:13.659946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e59c1b-083f-4041-82df-50a4a000e2bc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:13.723392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:13.757563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e59c1b-083f-4041-82df-50a4a000e2bc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:13.793255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422f7c2-b020-4026-9e27-e3c6db009401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:13.793904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422f7c2-b020-4026-9e27-e3c6db009401 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:13.794177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422f7c2-b020-4026-9e27-e3c6db009401 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:13.794464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422f7c2-b020-4026-9e27-e3c6db009401 None admin] PATH_INFO: `/v3/projects/b62fce02f29347a1b707951734566cc6/users/46216a7e489a4ec8bdd65975480d698c/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:13.796357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422f7c2-b020-4026-9e27-e3c6db009401 None admin] RBAC: Authorizing `identity:create_grant(project_id=b62fce02f29347a1b707951734566cc6, user_id=46216a7e489a4ec8bdd65975480d698c, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:13.797766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422f7c2-b020-4026-9e27-e3c6db009401 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:13.812383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 574/1153] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:50:13 2022] PUT /identity/v3/projects/b62fce02f29347a1b707951734566cc6/users/46216a7e489a4ec8bdd65975480d698c/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:50:15.724240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9198dd2-07e7-425b-a302-710db0d7849e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:15.724724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9198dd2-07e7-425b-a302-710db0d7849e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:15.724817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9198dd2-07e7-425b-a302-710db0d7849e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:15.726112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 580/1154] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:15.731279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac0bba-96e1-45bb-a4fd-f9934fb68abf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:15.731654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac0bba-96e1-45bb-a4fd-f9934fb68abf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:15.731838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac0bba-96e1-45bb-a4fd-f9934fb68abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:15.750118 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4bac0bba-96e1-45bb-a4fd-f9934fb68abf None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:15.751334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4bac0bba-96e1-45bb-a4fd-f9934fb68abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:15.815363 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 575/1155] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:16.087518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9198dd2-07e7-425b-a302-710db0d7849e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:16.188892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9198dd2-07e7-425b-a302-710db0d7849e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:16.201524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:16.222574 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f15f09-137d-4d9e-a1b9-270c724b9bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:16.223219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f15f09-137d-4d9e-a1b9-270c724b9bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:16.223501 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f15f09-137d-4d9e-a1b9-270c724b9bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:16.223773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f15f09-137d-4d9e-a1b9-270c724b9bc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:16.234845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f15f09-137d-4d9e-a1b9-270c724b9bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-38f15f09-137d-4d9e-a1b9-270c724b9bc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:50:16.239620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:50:16.240713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:50:16.241579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 581/1156] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:50:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:50:16.246760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac0bba-96e1-45bb-a4fd-f9934fb68abf None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:16.302686 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:16.345966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac0bba-96e1-45bb-a4fd-f9934fb68abf None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:16.378392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb9e9e-4b83-47bc-97a8-6174dd1ca17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:16.378987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb9e9e-4b83-47bc-97a8-6174dd1ca17a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:16.379292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb9e9e-4b83-47bc-97a8-6174dd1ca17a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:16.379563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb9e9e-4b83-47bc-97a8-6174dd1ca17a None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:16.380217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb9e9e-4b83-47bc-97a8-6174dd1ca17a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:16.380518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb9e9e-4b83-47bc-97a8-6174dd1ca17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:16.381869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb9e9e-4b83-47bc-97a8-6174dd1ca17a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:16.389630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 576/1157] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:50:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:16.395479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f15f09-137d-4d9e-a1b9-270c724b9bc9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:16.497118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f15f09-137d-4d9e-a1b9-270c724b9bc9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:16.529729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3593c-31fd-4718-abd6-8e75db046497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:16.530349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3593c-31fd-4718-abd6-8e75db046497 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:16.530629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3593c-31fd-4718-abd6-8e75db046497 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:16.530902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3593c-31fd-4718-abd6-8e75db046497 None admin] PATH_INFO: `/v3/users/46216a7e489a4ec8bdd65975480d698c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:16.532056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3593c-31fd-4718-abd6-8e75db046497 None admin] RBAC: Authorizing `identity:get_user(user_id=46216a7e489a4ec8bdd65975480d698c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:16.533484 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3593c-31fd-4718-abd6-8e75db046497 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:16.535075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 582/1158] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:50:16 2022] GET /identity/v3/users/46216a7e489a4ec8bdd65975480d698c => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:16.541033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb9e9e-4b83-47bc-97a8-6174dd1ca17a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:16.638486 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb9e9e-4b83-47bc-97a8-6174dd1ca17a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:16.654702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:16.671941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e5906-6039-4008-91c3-1aa35ef4cec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:16.672543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e5906-6039-4008-91c3-1aa35ef4cec8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:16.672822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e5906-6039-4008-91c3-1aa35ef4cec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:16.673096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e5906-6039-4008-91c3-1aa35ef4cec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:16.738847 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e5906-6039-4008-91c3-1aa35ef4cec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:16.740319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e5906-6039-4008-91c3-1aa35ef4cec8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:16.835289 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 577/1159] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:50:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:50:16.841317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3593c-31fd-4718-abd6-8e75db046497 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:16.882493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:16.939408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3593c-31fd-4718-abd6-8e75db046497 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:16.972048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3d381-aa81-4ed3-b0e3-f999de947001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:16.972666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3d381-aa81-4ed3-b0e3-f999de947001 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:16.972946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3d381-aa81-4ed3-b0e3-f999de947001 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:16.973216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3d381-aa81-4ed3-b0e3-f999de947001 None admin] PATH_INFO: `/v3/projects/b62fce02f29347a1b707951734566cc6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:16.974315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3d381-aa81-4ed3-b0e3-f999de947001 None admin] RBAC: Authorizing `identity:get_project(project_id=b62fce02f29347a1b707951734566cc6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:16.975756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3d381-aa81-4ed3-b0e3-f999de947001 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:16.977356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 583/1160] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:50:16 2022] GET /identity/v3/projects/b62fce02f29347a1b707951734566cc6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:16.984304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e5906-6039-4008-91c3-1aa35ef4cec8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:17.002093 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:17.082024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e5906-6039-4008-91c3-1aa35ef4cec8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:17.115820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85445a9a-1ba9-42a8-85e6-e34c4af3286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:17.116414 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85445a9a-1ba9-42a8-85e6-e34c4af3286f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:17.116696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85445a9a-1ba9-42a8-85e6-e34c4af3286f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:17.116970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85445a9a-1ba9-42a8-85e6-e34c4af3286f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:17.117690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85445a9a-1ba9-42a8-85e6-e34c4af3286f None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=b62fce02f29347a1b707951734566cc6, user.id=46216a7e489a4ec8bdd65975480d698c) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:17.117996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85445a9a-1ba9-42a8-85e6-e34c4af3286f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:17.119443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85445a9a-1ba9-42a8-85e6-e34c4af3286f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:17.125142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 578/1161] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:50:16 2022] GET /identity/v3/role_assignments?user.id=46216a7e489a4ec8bdd65975480d698c&scope.project.id=b62fce02f29347a1b707951734566cc6&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:19.059723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761033f-892d-4cf7-9c4a-c2efaeb2638c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:19.060264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761033f-892d-4cf7-9c4a-c2efaeb2638c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:19.060364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761033f-892d-4cf7-9c4a-c2efaeb2638c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:19.061623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 584/1162] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:19.066693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f83405-968c-47db-9ede-5c94a4cd41ce None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:19.067162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f83405-968c-47db-9ede-5c94a4cd41ce None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:19.067395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f83405-968c-47db-9ede-5c94a4cd41ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:19.085404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-77f83405-968c-47db-9ede-5c94a4cd41ce None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:19.086579 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-77f83405-968c-47db-9ede-5c94a4cd41ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:19.150357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 579/1163] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:19.421054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761033f-892d-4cf7-9c4a-c2efaeb2638c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:19.519438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761033f-892d-4cf7-9c4a-c2efaeb2638c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:19.528462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:19.553301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576fd34-55a8-4751-9503-95b3bc5962ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:19.553896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576fd34-55a8-4751-9503-95b3bc5962ee None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:19.554174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576fd34-55a8-4751-9503-95b3bc5962ee None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:19.554444 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576fd34-55a8-4751-9503-95b3bc5962ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:19.619263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576fd34-55a8-4751-9503-95b3bc5962ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:19.620716 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576fd34-55a8-4751-9503-95b3bc5962ee None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:19.715724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 585/1164] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:50:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:50:19.721831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f83405-968c-47db-9ede-5c94a4cd41ce None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:19.735956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:19.818826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f83405-968c-47db-9ede-5c94a4cd41ce None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:19.850668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee9f91-ad11-415d-85c9-109ff51cde17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:19.851307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee9f91-ad11-415d-85c9-109ff51cde17 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:19.851585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee9f91-ad11-415d-85c9-109ff51cde17 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:19.851853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee9f91-ad11-415d-85c9-109ff51cde17 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:19.858696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee9f91-ad11-415d-85c9-109ff51cde17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:19.860145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee9f91-ad11-415d-85c9-109ff51cde17 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-64ee9f91-ad11-415d-85c9-109ff51cde17 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:50:19.871668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:50:19.872894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:50:19.874014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 580/1165] 10.222.0.86 () {56 vars in 1117 bytes} [Mon Aug 29 18:50:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:50:19.878689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576fd34-55a8-4751-9503-95b3bc5962ee None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:19.888611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:19.976014 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576fd34-55a8-4751-9503-95b3bc5962ee None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:20.009421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b7069-c3ef-4288-ab56-dd58b49f7cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:20.010001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7069-c3ef-4288-ab56-dd58b49f7cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:20.010274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7069-c3ef-4288-ab56-dd58b49f7cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:20.010562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7069-c3ef-4288-ab56-dd58b49f7cc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:20.011208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b7069-c3ef-4288-ab56-dd58b49f7cc2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:20.011505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b7069-c3ef-4288-ab56-dd58b49f7cc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:20.012844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b7069-c3ef-4288-ab56-dd58b49f7cc2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:20.019647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 586/1166] 10.222.0.86 () {56 vars in 1115 bytes} [Mon Aug 29 18:50:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:21.904233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56465a2c-82ef-44b9-9796-67724f97ba17 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:21.904775 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56465a2c-82ef-44b9-9796-67724f97ba17 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:21.904775 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56465a2c-82ef-44b9-9796-67724f97ba17 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:21.906099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 581/1167] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:21.911217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd48dfd-fa30-4e22-a1a5-097cc60f1635 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:21.911608 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd48dfd-fa30-4e22-a1a5-097cc60f1635 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:21.911764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd48dfd-fa30-4e22-a1a5-097cc60f1635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:21.930006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-afd48dfd-fa30-4e22-a1a5-097cc60f1635 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:21.931215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-afd48dfd-fa30-4e22-a1a5-097cc60f1635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:22.001287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 587/1168] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:22.289567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56465a2c-82ef-44b9-9796-67724f97ba17 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:22.359824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:22.390238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56465a2c-82ef-44b9-9796-67724f97ba17 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:22.425743 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685ae8b-504a-48b3-9c06-199797f4fbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:22.426349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685ae8b-504a-48b3-9c06-199797f4fbdc None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:22.426628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685ae8b-504a-48b3-9c06-199797f4fbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:22.426897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685ae8b-504a-48b3-9c06-199797f4fbdc None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:22.427595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685ae8b-504a-48b3-9c06-199797f4fbdc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:22.428952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685ae8b-504a-48b3-9c06-199797f4fbdc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:22.444238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 582/1169] 10.222.0.86 () {60 vars in 1138 bytes} [Mon Aug 29 18:50:22 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:50:24.380385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1804a-e2ff-475c-aaa1-2036ed2f92e7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:24.380934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1804a-e2ff-475c-aaa1-2036ed2f92e7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:24.380934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1804a-e2ff-475c-aaa1-2036ed2f92e7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:24.382267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 588/1170] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:24.387428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1044b-e66a-401a-9a65-2eef93eb3d5f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:24.387863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1044b-e66a-401a-9a65-2eef93eb3d5f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:24.388007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1044b-e66a-401a-9a65-2eef93eb3d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:24.405707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9cb1044b-e66a-401a-9a65-2eef93eb3d5f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:24.406884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9cb1044b-e66a-401a-9a65-2eef93eb3d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:24.470781 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 583/1171] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:24.734492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1804a-e2ff-475c-aaa1-2036ed2f92e7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:24.737415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:24.833218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1804a-e2ff-475c-aaa1-2036ed2f92e7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:24.865732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed7e55-4202-4fc5-9891-2abb4be1fbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:24.866346 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed7e55-4202-4fc5-9891-2abb4be1fbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:24.866623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed7e55-4202-4fc5-9891-2abb4be1fbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:24.866892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed7e55-4202-4fc5-9891-2abb4be1fbb4 None admin] PATH_INFO: `/v3/domains/b884925b2b27480a926af22d8eaf3ebc` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:24.876913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed7e55-4202-4fc5-9891-2abb4be1fbb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b884925b2b27480a926af22d8eaf3ebc)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:24.878340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed7e55-4202-4fc5-9891-2abb4be1fbb4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:24.879923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 589/1172] 10.222.0.86 () {56 vars in 1183 bytes} [Mon Aug 29 18:50:24 2022] GET /identity/v3/domains/b884925b2b27480a926af22d8eaf3ebc => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:24.886277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1044b-e66a-401a-9a65-2eef93eb3d5f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:24.948791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:24.987272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1044b-e66a-401a-9a65-2eef93eb3d5f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:25.021013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd783a32-6b53-4d53-9401-747387cffc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:25.021608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd783a32-6b53-4d53-9401-747387cffc53 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:25.021885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd783a32-6b53-4d53-9401-747387cffc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:25.022154 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd783a32-6b53-4d53-9401-747387cffc53 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:25.022801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd783a32-6b53-4d53-9401-747387cffc53 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:25.024246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd783a32-6b53-4d53-9401-747387cffc53 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:25.050556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 584/1173] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:50:24 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:50:26.978697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ec2c3-b47f-410e-9007-5e1bf7d96fde None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:26.979228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ec2c3-b47f-410e-9007-5e1bf7d96fde None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:26.979228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ec2c3-b47f-410e-9007-5e1bf7d96fde None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:26.980546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 590/1174] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:26.985733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f1425-2012-4058-aefc-9a7648f000be None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:26.986172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f1425-2012-4058-aefc-9a7648f000be None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:26.986289 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f1425-2012-4058-aefc-9a7648f000be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:27.004760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-dd0f1425-2012-4058-aefc-9a7648f000be None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:27.005942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dd0f1425-2012-4058-aefc-9a7648f000be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:27.070515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 585/1175] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:27.339879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ec2c3-b47f-410e-9007-5e1bf7d96fde None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:27.401659 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:27.439665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ec2c3-b47f-410e-9007-5e1bf7d96fde None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:27.473496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1f28b-83c8-4c67-a187-4eb622bdc840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:27.474108 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1f28b-83c8-4c67-a187-4eb622bdc840 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:27.474387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1f28b-83c8-4c67-a187-4eb622bdc840 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:27.474656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1f28b-83c8-4c67-a187-4eb622bdc840 None admin] PATH_INFO: `/v3/domains/b884925b2b27480a926af22d8eaf3ebc` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:27.475785 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1f28b-83c8-4c67-a187-4eb622bdc840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b884925b2b27480a926af22d8eaf3ebc)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:27.477207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1f28b-83c8-4c67-a187-4eb622bdc840 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:27.478738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 591/1176] 10.222.0.86 () {56 vars in 1183 bytes} [Mon Aug 29 18:50:27 2022] GET /identity/v3/domains/b884925b2b27480a926af22d8eaf3ebc => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:27.484668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f1425-2012-4058-aefc-9a7648f000be None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:27.528832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:27.581435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f1425-2012-4058-aefc-9a7648f000be None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:27.613764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cadf6-3e04-4488-be39-36f9e8596348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:27.614295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cadf6-3e04-4488-be39-36f9e8596348 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:27.614571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cadf6-3e04-4488-be39-36f9e8596348 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:27.614842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cadf6-3e04-4488-be39-36f9e8596348 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:27.615518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cadf6-3e04-4488-be39-36f9e8596348 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:27.616937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cadf6-3e04-4488-be39-36f9e8596348 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:27.638680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 586/1177] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:50:27 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:50:29.577596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e88de-5ca4-4b65-b046-842f1687a285 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:29.578233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e88de-5ca4-4b65-b046-842f1687a285 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:29.578233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e88de-5ca4-4b65-b046-842f1687a285 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:29.579507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 592/1178] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:29.584695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393cbdd-1e54-4723-933d-6003668e6467 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:29.585142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393cbdd-1e54-4723-933d-6003668e6467 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:29.585246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393cbdd-1e54-4723-933d-6003668e6467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:29.604547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a393cbdd-1e54-4723-933d-6003668e6467 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:29.605772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a393cbdd-1e54-4723-933d-6003668e6467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:29.674707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 587/1179] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:29.946842 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e88de-5ca4-4b65-b046-842f1687a285 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:30.059424 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e88de-5ca4-4b65-b046-842f1687a285 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:30.088280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:30.095008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4be14-b2fa-43d0-b351-38ae2a58ac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:30.095653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4be14-b2fa-43d0-b351-38ae2a58ac3a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:30.095932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4be14-b2fa-43d0-b351-38ae2a58ac3a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:30.096199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4be14-b2fa-43d0-b351-38ae2a58ac3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:30.106473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4be14-b2fa-43d0-b351-38ae2a58ac3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-dcf4be14-b2fa-43d0-b351-38ae2a58ac3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:50:30.111227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:50:30.112412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:50:30.113594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 593/1180] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:50:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:50:30.119315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393cbdd-1e54-4723-933d-6003668e6467 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:30.148716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:30.217149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393cbdd-1e54-4723-933d-6003668e6467 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:30.248910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9b2fa-c0b7-42ae-b90c-9618df2df875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:30.249504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b2fa-c0b7-42ae-b90c-9618df2df875 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:30.249783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b2fa-c0b7-42ae-b90c-9618df2df875 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:30.250055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b2fa-c0b7-42ae-b90c-9618df2df875 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:30.250710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9b2fa-c0b7-42ae-b90c-9618df2df875 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:30.251034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9b2fa-c0b7-42ae-b90c-9618df2df875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:30.252387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9b2fa-c0b7-42ae-b90c-9618df2df875 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:30.260077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 588/1181] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:50:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:30.266509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4be14-b2fa-43d0-b351-38ae2a58ac3a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:30.373099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4be14-b2fa-43d0-b351-38ae2a58ac3a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:30.407977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d779e0-f19e-4a81-9cfe-82f2b2819843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:30.408598 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d779e0-f19e-4a81-9cfe-82f2b2819843 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:30.408875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d779e0-f19e-4a81-9cfe-82f2b2819843 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:30.409145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d779e0-f19e-4a81-9cfe-82f2b2819843 None admin] PATH_INFO: `/v3/users/958a55b8567e47468eedc7b079fbf1f2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:30.423130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d779e0-f19e-4a81-9cfe-82f2b2819843 None admin] RBAC: Authorizing `identity:get_user(user_id=958a55b8567e47468eedc7b079fbf1f2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:30.424564 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d779e0-f19e-4a81-9cfe-82f2b2819843 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:30.426161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 594/1182] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:50:30 2022] GET /identity/v3/users/958a55b8567e47468eedc7b079fbf1f2 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:30.432107 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9b2fa-c0b7-42ae-b90c-9618df2df875 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:30.521760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:30.530847 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9b2fa-c0b7-42ae-b90c-9618df2df875 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:30.563315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521274b2-6dcd-4aa2-99b8-c66412ca5fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:30.563913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521274b2-6dcd-4aa2-99b8-c66412ca5fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:30.564194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521274b2-6dcd-4aa2-99b8-c66412ca5fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:30.564465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521274b2-6dcd-4aa2-99b8-c66412ca5fa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:30.629606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521274b2-6dcd-4aa2-99b8-c66412ca5fa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:30.631053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521274b2-6dcd-4aa2-99b8-c66412ca5fa0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:30.720230 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:30.741330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 589/1183] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:50:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:50:30.747872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d779e0-f19e-4a81-9cfe-82f2b2819843 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:30.795087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:30.844841 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d779e0-f19e-4a81-9cfe-82f2b2819843 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:30.878114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a6928-fdfa-4234-901f-d987b387d036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:30.878723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a6928-fdfa-4234-901f-d987b387d036 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:30.879028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a6928-fdfa-4234-901f-d987b387d036 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:30.879297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a6928-fdfa-4234-901f-d987b387d036 None admin] PATH_INFO: `/v3/projects/5c4c8804bde64b92b0d601553dbdc033` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:30.880382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a6928-fdfa-4234-901f-d987b387d036 None admin] RBAC: Authorizing `identity:get_project(project_id=5c4c8804bde64b92b0d601553dbdc033)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:30.881790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a6928-fdfa-4234-901f-d987b387d036 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:30.883408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 595/1184] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:50:30 2022] GET /identity/v3/projects/5c4c8804bde64b92b0d601553dbdc033 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:30.890796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521274b2-6dcd-4aa2-99b8-c66412ca5fa0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:30.986796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521274b2-6dcd-4aa2-99b8-c66412ca5fa0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:31.019646 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81f016-8e95-478c-a09e-5589751b7417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:31.020246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81f016-8e95-478c-a09e-5589751b7417 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:31.020529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81f016-8e95-478c-a09e-5589751b7417 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:31.020803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81f016-8e95-478c-a09e-5589751b7417 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:31.021562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81f016-8e95-478c-a09e-5589751b7417 None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=5c4c8804bde64b92b0d601553dbdc033, user.id=958a55b8567e47468eedc7b079fbf1f2) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:31.021854 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81f016-8e95-478c-a09e-5589751b7417 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:31.023300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81f016-8e95-478c-a09e-5589751b7417 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:31.028510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 590/1185] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:50:30 2022] GET /identity/v3/role_assignments?user.id=958a55b8567e47468eedc7b079fbf1f2&scope.project.id=5c4c8804bde64b92b0d601553dbdc033&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:32.897839 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b6501-f39f-4b5d-82f5-da9c43a26a17 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:32.898399 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b6501-f39f-4b5d-82f5-da9c43a26a17 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:32.898399 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b6501-f39f-4b5d-82f5-da9c43a26a17 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:32.899732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 596/1186] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:32.904829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490d66d-36e0-4aa2-8926-628ab274353d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:32.905245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490d66d-36e0-4aa2-8926-628ab274353d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:32.905356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490d66d-36e0-4aa2-8926-628ab274353d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:32.923145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1490d66d-36e0-4aa2-8926-628ab274353d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:32.924321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1490d66d-36e0-4aa2-8926-628ab274353d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:32.972457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:32.989612 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 591/1187] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:33.268791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b6501-f39f-4b5d-82f5-da9c43a26a17 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:33.367878 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b6501-f39f-4b5d-82f5-da9c43a26a17 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:33.384240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:33.403460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee9a88-108d-480f-9bf5-0fcff1a2b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:33.404092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee9a88-108d-480f-9bf5-0fcff1a2b4ab None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:33.404373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee9a88-108d-480f-9bf5-0fcff1a2b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:33.404662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee9a88-108d-480f-9bf5-0fcff1a2b4ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:33.414426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee9a88-108d-480f-9bf5-0fcff1a2b4ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-dbee9a88-108d-480f-9bf5-0fcff1a2b4ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:50:33.419320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:50:33.420472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:50:33.421503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 597/1188] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:50:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:50:33.433034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490d66d-36e0-4aa2-8926-628ab274353d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:33.536375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490d66d-36e0-4aa2-8926-628ab274353d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:33.568453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f527284-d20b-4b23-a11f-2b9b98206a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:33.569080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f527284-d20b-4b23-a11f-2b9b98206a90 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:33.569355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f527284-d20b-4b23-a11f-2b9b98206a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:33.569626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f527284-d20b-4b23-a11f-2b9b98206a90 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:33.570536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f527284-d20b-4b23-a11f-2b9b98206a90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:33.570833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f527284-d20b-4b23-a11f-2b9b98206a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:33.572204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f527284-d20b-4b23-a11f-2b9b98206a90 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:33.580451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 592/1189] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:50:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:33.586235 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee9a88-108d-480f-9bf5-0fcff1a2b4ab None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:33.682009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee9a88-108d-480f-9bf5-0fcff1a2b4ab None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:33.716013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537adf97-2378-45e0-81e2-65cdb2f2dbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:33.716628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537adf97-2378-45e0-81e2-65cdb2f2dbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:33.716907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537adf97-2378-45e0-81e2-65cdb2f2dbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:33.717178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537adf97-2378-45e0-81e2-65cdb2f2dbf8 None admin] PATH_INFO: `/v3/users/958a55b8567e47468eedc7b079fbf1f2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:33.718278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537adf97-2378-45e0-81e2-65cdb2f2dbf8 None admin] RBAC: Authorizing `identity:get_user(user_id=958a55b8567e47468eedc7b079fbf1f2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:33.719721 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537adf97-2378-45e0-81e2-65cdb2f2dbf8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:33.721286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 598/1190] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:50:33 2022] GET /identity/v3/users/958a55b8567e47468eedc7b079fbf1f2 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:33.727031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f527284-d20b-4b23-a11f-2b9b98206a90 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:33.775067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:33.824635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f527284-d20b-4b23-a11f-2b9b98206a90 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:33.856583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ea2fb-c569-4568-b150-0f6663abf3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:33.857192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ea2fb-c569-4568-b150-0f6663abf3ae None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:33.857470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ea2fb-c569-4568-b150-0f6663abf3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:33.857744 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ea2fb-c569-4568-b150-0f6663abf3ae None admin] PATH_INFO: `/v3/projects/5c4c8804bde64b92b0d601553dbdc033` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:33.858843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ea2fb-c569-4568-b150-0f6663abf3ae None admin] RBAC: Authorizing `identity:get_project(project_id=5c4c8804bde64b92b0d601553dbdc033)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:33.860281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ea2fb-c569-4568-b150-0f6663abf3ae None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:33.861869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 593/1191] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:50:33 2022] GET /identity/v3/projects/5c4c8804bde64b92b0d601553dbdc033 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:33.867812 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537adf97-2378-45e0-81e2-65cdb2f2dbf8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:33.912481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:33.966003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537adf97-2378-45e0-81e2-65cdb2f2dbf8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:33.999404 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d72fd6-2154-4c45-858b-e076eb0a67fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:34.000049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d72fd6-2154-4c45-858b-e076eb0a67fe None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:34.000330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d72fd6-2154-4c45-858b-e076eb0a67fe None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:34.000619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d72fd6-2154-4c45-858b-e076eb0a67fe None admin] PATH_INFO: `/v3/projects/5c4c8804bde64b92b0d601553dbdc033/users/958a55b8567e47468eedc7b079fbf1f2/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:34.002487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d72fd6-2154-4c45-858b-e076eb0a67fe None admin] RBAC: Authorizing `identity:create_grant(project_id=5c4c8804bde64b92b0d601553dbdc033, user_id=958a55b8567e47468eedc7b079fbf1f2, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:34.003938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d72fd6-2154-4c45-858b-e076eb0a67fe None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:34.016461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 599/1192] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:50:33 2022] PUT /identity/v3/projects/5c4c8804bde64b92b0d601553dbdc033/users/958a55b8567e47468eedc7b079fbf1f2/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:50:35.906247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d0d69-1143-4d77-a43e-368485f1306b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:35.906766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d0d69-1143-4d77-a43e-368485f1306b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:35.906766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d0d69-1143-4d77-a43e-368485f1306b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:35.908162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 594/1193] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:50:35.913392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80af05-14c3-4278-99d8-9e1551419d01 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:35.913791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80af05-14c3-4278-99d8-9e1551419d01 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:35.913947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80af05-14c3-4278-99d8-9e1551419d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:35.931939 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-dd80af05-14c3-4278-99d8-9e1551419d01 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:50:35.933655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dd80af05-14c3-4278-99d8-9e1551419d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:50:35.999222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 600/1194] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:50:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:50:36.265653 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d0d69-1143-4d77-a43e-368485f1306b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:36.364076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d0d69-1143-4d77-a43e-368485f1306b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:36.379723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:36.396823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57df554-74a3-4924-8714-00e4df2509d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:36.397437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57df554-74a3-4924-8714-00e4df2509d2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:36.397714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57df554-74a3-4924-8714-00e4df2509d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:36.397981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57df554-74a3-4924-8714-00e4df2509d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:36.408855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57df554-74a3-4924-8714-00e4df2509d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f57df554-74a3-4924-8714-00e4df2509d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:50:36.413552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:50:36.414490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:50:36.415458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 595/1195] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:50:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:50:36.421324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80af05-14c3-4278-99d8-9e1551419d01 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:36.454363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:36.525215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80af05-14c3-4278-99d8-9e1551419d01 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:36.558861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9535820-4e04-44d3-8132-d5e76e6eb37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:36.559489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9535820-4e04-44d3-8132-d5e76e6eb37c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:36.559776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9535820-4e04-44d3-8132-d5e76e6eb37c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:36.560041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9535820-4e04-44d3-8132-d5e76e6eb37c None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:36.560691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9535820-4e04-44d3-8132-d5e76e6eb37c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:36.560989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9535820-4e04-44d3-8132-d5e76e6eb37c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:36.562458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9535820-4e04-44d3-8132-d5e76e6eb37c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:36.570485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 601/1196] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:50:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:36.576309 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57df554-74a3-4924-8714-00e4df2509d2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:36.671375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57df554-74a3-4924-8714-00e4df2509d2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:36.703747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46b0fc-31bc-4826-9295-c9ff1573f195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:36.704367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46b0fc-31bc-4826-9295-c9ff1573f195 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:36.704641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46b0fc-31bc-4826-9295-c9ff1573f195 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:36.704909 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46b0fc-31bc-4826-9295-c9ff1573f195 None admin] PATH_INFO: `/v3/users/958a55b8567e47468eedc7b079fbf1f2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:36.705994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46b0fc-31bc-4826-9295-c9ff1573f195 None admin] RBAC: Authorizing `identity:get_user(user_id=958a55b8567e47468eedc7b079fbf1f2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:36.707435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46b0fc-31bc-4826-9295-c9ff1573f195 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:36.708983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 596/1197] 10.222.0.86 () {56 vars in 1177 bytes} [Mon Aug 29 18:50:36 2022] GET /identity/v3/users/958a55b8567e47468eedc7b079fbf1f2 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:36.715350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9535820-4e04-44d3-8132-d5e76e6eb37c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:36.806279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:36.812413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9535820-4e04-44d3-8132-d5e76e6eb37c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:36.845228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc96c52-fd54-4295-9bb2-ee3aa5116c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:36.845819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc96c52-fd54-4295-9bb2-ee3aa5116c1f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:36.846093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc96c52-fd54-4295-9bb2-ee3aa5116c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:36.846360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc96c52-fd54-4295-9bb2-ee3aa5116c1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:36.910756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc96c52-fd54-4295-9bb2-ee3aa5116c1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:36.912227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc96c52-fd54-4295-9bb2-ee3aa5116c1f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:37.009143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:37.016097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 602/1198] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:50:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:50:37.022442 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46b0fc-31bc-4826-9295-c9ff1573f195 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:37.067545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:37.121165 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46b0fc-31bc-4826-9295-c9ff1573f195 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:37.152795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432583d-6496-4683-ac48-26d65ecfadb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:37.153434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432583d-6496-4683-ac48-26d65ecfadb2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:37.153712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432583d-6496-4683-ac48-26d65ecfadb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:37.153984 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432583d-6496-4683-ac48-26d65ecfadb2 None admin] PATH_INFO: `/v3/projects/5c4c8804bde64b92b0d601553dbdc033` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:37.155148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432583d-6496-4683-ac48-26d65ecfadb2 None admin] RBAC: Authorizing `identity:get_project(project_id=5c4c8804bde64b92b0d601553dbdc033)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:37.156593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432583d-6496-4683-ac48-26d65ecfadb2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:37.158178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 597/1199] 10.222.0.86 () {56 vars in 1186 bytes} [Mon Aug 29 18:50:37 2022] GET /identity/v3/projects/5c4c8804bde64b92b0d601553dbdc033 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:37.164437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc96c52-fd54-4295-9bb2-ee3aa5116c1f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:50:37.261681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc96c52-fd54-4295-9bb2-ee3aa5116c1f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:50:37.294585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4cc8f-ab24-4d93-8e3f-96a3de2c25e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:50:37.295203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4cc8f-ab24-4d93-8e3f-96a3de2c25e4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:50:37.295474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4cc8f-ab24-4d93-8e3f-96a3de2c25e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:50:37.295738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4cc8f-ab24-4d93-8e3f-96a3de2c25e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:50:37.296444 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4cc8f-ab24-4d93-8e3f-96a3de2c25e4 None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=5c4c8804bde64b92b0d601553dbdc033, user.id=958a55b8567e47468eedc7b079fbf1f2) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:50:37.296747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4cc8f-ab24-4d93-8e3f-96a3de2c25e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:50:37.298159 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4cc8f-ab24-4d93-8e3f-96a3de2c25e4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:50:37.303698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 603/1200] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:50:37 2022] GET /identity/v3/role_assignments?user.id=958a55b8567e47468eedc7b079fbf1f2&scope.project.id=5c4c8804bde64b92b0d601553dbdc033&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:50:47.148722 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=44 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:50:47.301169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=70 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:32.953827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade1ea-e71b-490c-8c20-f47cb9f80be5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:32.954449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade1ea-e71b-490c-8c20-f47cb9f80be5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:32.954449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade1ea-e71b-490c-8c20-f47cb9f80be5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:32.955676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 598/1201] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:51:32.961212 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2b016-b3e9-4386-8213-730f0ed289f2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:32.961703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2b016-b3e9-4386-8213-730f0ed289f2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:32.961788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2b016-b3e9-4386-8213-730f0ed289f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:32.984471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-83b2b016-b3e9-4386-8213-730f0ed289f2 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:51:32.985874 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-83b2b016-b3e9-4386-8213-730f0ed289f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:51:33.078430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 604/1202] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:51:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:51:33.370870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ade1ea-e71b-490c-8c20-f47cb9f80be5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:33.519034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ade1ea-e71b-490c-8c20-f47cb9f80be5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:33.562499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfa013-de56-493f-bcb5-b1a094b04b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:33.563215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfa013-de56-493f-bcb5-b1a094b04b21 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:33.563535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfa013-de56-493f-bcb5-b1a094b04b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:33.563785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfa013-de56-493f-bcb5-b1a094b04b21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:33.579283 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcfa013-de56-493f-bcb5-b1a094b04b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:33.580854 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcfa013-de56-493f-bcb5-b1a094b04b21 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0bcfa013-de56-493f-bcb5-b1a094b04b21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:33.599052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:33.600339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:33.601515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:33.601515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:33.601515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:33.603084 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 599/1203] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:33.609850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2b016-b3e9-4386-8213-730f0ed289f2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:33.734949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2b016-b3e9-4386-8213-730f0ed289f2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:33.768371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:33.776393 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8dbc7-890b-4c93-9493-1e61cbac290d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:33.777563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8dbc7-890b-4c93-9493-1e61cbac290d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:33.777809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8dbc7-890b-4c93-9493-1e61cbac290d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:33.778161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8dbc7-890b-4c93-9493-1e61cbac290d None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:33.778812 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8dbc7-890b-4c93-9493-1e61cbac290d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:33.779333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8dbc7-890b-4c93-9493-1e61cbac290d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:33.781239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8dbc7-890b-4c93-9493-1e61cbac290d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:33.800240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 605/1204] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:33.802089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfa013-de56-493f-bcb5-b1a094b04b21 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:33.877088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:33.933693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfa013-de56-493f-bcb5-b1a094b04b21 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:33.976728 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680205c9-ddb3-4cf0-973b-f6d32a9f7f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:33.977356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680205c9-ddb3-4cf0-973b-f6d32a9f7f13 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:33.977603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680205c9-ddb3-4cf0-973b-f6d32a9f7f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:33.977808 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680205c9-ddb3-4cf0-973b-f6d32a9f7f13 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:33.978454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680205c9-ddb3-4cf0-973b-f6d32a9f7f13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:33.983026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680205c9-ddb3-4cf0-973b-f6d32a9f7f13 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:34.005213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 600/1205] 10.222.0.86 () {60 vars in 1132 bytes} [Mon Aug 29 18:51:33 2022] POST /identity/v3/users => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:51:36.080681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbc964-98a7-4aba-bfeb-f43a1e12cb98 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:36.081234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbc964-98a7-4aba-bfeb-f43a1e12cb98 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:36.081234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbc964-98a7-4aba-bfeb-f43a1e12cb98 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:36.082690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 606/1206] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:51:36.088311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aaad9-3a4c-4e14-ba5b-9ff8f927d409 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:36.090698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aaad9-3a4c-4e14-ba5b-9ff8f927d409 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:36.090698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aaad9-3a4c-4e14-ba5b-9ff8f927d409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:36.109943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-771aaad9-3a4c-4e14-ba5b-9ff8f927d409 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:51:36.111162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-771aaad9-3a4c-4e14-ba5b-9ff8f927d409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:51:36.190337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 601/1207] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:51:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:51:36.481904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbc964-98a7-4aba-bfeb-f43a1e12cb98 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:36.602006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbc964-98a7-4aba-bfeb-f43a1e12cb98 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:36.643418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50499556-34b4-4d91-a0c1-5d2bf66e417d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:36.644321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50499556-34b4-4d91-a0c1-5d2bf66e417d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:36.644697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50499556-34b4-4d91-a0c1-5d2bf66e417d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:36.645083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50499556-34b4-4d91-a0c1-5d2bf66e417d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:36.659466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50499556-34b4-4d91-a0c1-5d2bf66e417d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-50499556-34b4-4d91-a0c1-5d2bf66e417d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:51:36.665056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:36.666306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:36.668185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 607/1208] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:51:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:36.674548 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771aaad9-3a4c-4e14-ba5b-9ff8f927d409 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:36.721196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:36.793869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771aaad9-3a4c-4e14-ba5b-9ff8f927d409 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:36.833073 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc5991-86b9-4005-ad6a-c25dfcda42ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:36.833793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc5991-86b9-4005-ad6a-c25dfcda42ad None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:36.834161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc5991-86b9-4005-ad6a-c25dfcda42ad None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:36.834436 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc5991-86b9-4005-ad6a-c25dfcda42ad None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:36.835139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc5991-86b9-4005-ad6a-c25dfcda42ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:36.835453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc5991-86b9-4005-ad6a-c25dfcda42ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:36.836933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc5991-86b9-4005-ad6a-c25dfcda42ad None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:36.848198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 602/1209] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:51:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:51:36.852682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50499556-34b4-4d91-a0c1-5d2bf66e417d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:36.920866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:36.991926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50499556-34b4-4d91-a0c1-5d2bf66e417d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:37.037683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529d8c2-4a78-4222-9212-a6141cc56159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:37.038316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529d8c2-4a78-4222-9212-a6141cc56159 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:37.038591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529d8c2-4a78-4222-9212-a6141cc56159 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:37.038860 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529d8c2-4a78-4222-9212-a6141cc56159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:37.052245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529d8c2-4a78-4222-9212-a6141cc56159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:37.053747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529d8c2-4a78-4222-9212-a6141cc56159 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5529d8c2-4a78-4222-9212-a6141cc56159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:37.072345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:37.074034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:37.074924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:37.074924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:37.074924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:37.074924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 608/1210] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:37.080941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc5991-86b9-4005-ad6a-c25dfcda42ad None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:37.203777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:37.207121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc5991-86b9-4005-ad6a-c25dfcda42ad None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:37.248643 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4391c-13a8-48f1-80c8-6978ec46ec2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:37.249399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4391c-13a8-48f1-80c8-6978ec46ec2b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:37.249750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4391c-13a8-48f1-80c8-6978ec46ec2b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:37.250069 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4391c-13a8-48f1-80c8-6978ec46ec2b None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:37.251007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4391c-13a8-48f1-80c8-6978ec46ec2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:37.251319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4391c-13a8-48f1-80c8-6978ec46ec2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:37.252845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4391c-13a8-48f1-80c8-6978ec46ec2b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:37.266431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 603/1211] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:37.272227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529d8c2-4a78-4222-9212-a6141cc56159 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:37.396638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529d8c2-4a78-4222-9212-a6141cc56159 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:37.398638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:37.435472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbeb9ea-2295-4e9b-b9a9-c0f18e847f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:37.436172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeb9ea-2295-4e9b-b9a9-c0f18e847f93 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:37.436428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeb9ea-2295-4e9b-b9a9-c0f18e847f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:37.436698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeb9ea-2295-4e9b-b9a9-c0f18e847f93 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:37.445053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbeb9ea-2295-4e9b-b9a9-c0f18e847f93 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:37.446967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbeb9ea-2295-4e9b-b9a9-c0f18e847f93 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-bcbeb9ea-2295-4e9b-b9a9-c0f18e847f93 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:51:37.459106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:37.460311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:37.461287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:37.461287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:37.461287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:51:37.461287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:51:37.461287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:51:37.461287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:51:37.461287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:51:37.461287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:51:37.461287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:51:37.461287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:37.461287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 609/1212] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:37.467562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4391c-13a8-48f1-80c8-6978ec46ec2b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:37.590311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4391c-13a8-48f1-80c8-6978ec46ec2b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:37.630913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1718ab-d203-4373-adc5-b5fbf09e2447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:37.631579 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1718ab-d203-4373-adc5-b5fbf09e2447 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:37.631946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1718ab-d203-4373-adc5-b5fbf09e2447 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:37.633519 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1718ab-d203-4373-adc5-b5fbf09e2447 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:37.633519 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1718ab-d203-4373-adc5-b5fbf09e2447 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:37.633519 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1718ab-d203-4373-adc5-b5fbf09e2447 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:37.634719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1718ab-d203-4373-adc5-b5fbf09e2447 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:37.654886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 604/1213] 10.222.0.86 () {56 vars in 1143 bytes} [Mon Aug 29 18:51:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:37.660230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbeb9ea-2295-4e9b-b9a9-c0f18e847f93 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:37.779137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbeb9ea-2295-4e9b-b9a9-c0f18e847f93 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:37.821512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b0a81-6760-4c6c-92dc-5490e8071747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:37.822281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b0a81-6760-4c6c-92dc-5490e8071747 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:37.822654 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b0a81-6760-4c6c-92dc-5490e8071747 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:37.823569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b0a81-6760-4c6c-92dc-5490e8071747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:37.861173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:37.909384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b0a81-6760-4c6c-92dc-5490e8071747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:37.909384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b0a81-6760-4c6c-92dc-5490e8071747 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:38.030033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 610/1214] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:51:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 18:51:38.035927 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1718ab-d203-4373-adc5-b5fbf09e2447 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:38.057052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:38.159856 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1718ab-d203-4373-adc5-b5fbf09e2447 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:38.202052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab711f9-3e30-4a9d-b6d2-9d08b6fb793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:38.202740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab711f9-3e30-4a9d-b6d2-9d08b6fb793e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:38.202989 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab711f9-3e30-4a9d-b6d2-9d08b6fb793e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:38.204831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab711f9-3e30-4a9d-b6d2-9d08b6fb793e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:38.214837 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab711f9-3e30-4a9d-b6d2-9d08b6fb793e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:38.216602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab711f9-3e30-4a9d-b6d2-9d08b6fb793e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-dab711f9-3e30-4a9d-b6d2-9d08b6fb793e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:38.232872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:38.234159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:38.235268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:38.235268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:38.235268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:38.238809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 605/1215] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:38.239946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b0a81-6760-4c6c-92dc-5490e8071747 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:38.305040 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:38.352786 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b0a81-6760-4c6c-92dc-5490e8071747 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:38.390584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd9c2a-5543-40d8-a264-20c8f0006d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:38.391222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd9c2a-5543-40d8-a264-20c8f0006d8b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:38.391506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd9c2a-5543-40d8-a264-20c8f0006d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:38.391779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd9c2a-5543-40d8-a264-20c8f0006d8b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:38.392430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd9c2a-5543-40d8-a264-20c8f0006d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:38.392738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd9c2a-5543-40d8-a264-20c8f0006d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:38.394094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd9c2a-5543-40d8-a264-20c8f0006d8b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:38.406577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 611/1216] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:38.412372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab711f9-3e30-4a9d-b6d2-9d08b6fb793e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:38.491429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:38.533929 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab711f9-3e30-4a9d-b6d2-9d08b6fb793e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:38.576733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4df3c-b8c6-47f7-8809-c7cba7c7e97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:38.577472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4df3c-b8c6-47f7-8809-c7cba7c7e97f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:38.577848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4df3c-b8c6-47f7-8809-c7cba7c7e97f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:38.578175 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4df3c-b8c6-47f7-8809-c7cba7c7e97f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:38.586925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4df3c-b8c6-47f7-8809-c7cba7c7e97f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:38.588773 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4df3c-b8c6-47f7-8809-c7cba7c7e97f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-57a4df3c-b8c6-47f7-8809-c7cba7c7e97f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:51:38.602374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:38.603871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:38.605372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:51:38.605372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:51:38.605372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:51:38.605372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:51:38.605372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:51:38.605372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:51:38.605372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:51:38.605372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:38.607160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 606/1217] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:51:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:38.609910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd9c2a-5543-40d8-a264-20c8f0006d8b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:38.722149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd9c2a-5543-40d8-a264-20c8f0006d8b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:38.746966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:38.759499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530701ff-6187-4614-9566-a56824318530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:38.760109 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530701ff-6187-4614-9566-a56824318530 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:38.760383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530701ff-6187-4614-9566-a56824318530 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:38.760650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530701ff-6187-4614-9566-a56824318530 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:38.761333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530701ff-6187-4614-9566-a56824318530 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:38.761753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530701ff-6187-4614-9566-a56824318530 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:38.763200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530701ff-6187-4614-9566-a56824318530 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:38.781452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 612/1218] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:51:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:38.782651 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4df3c-b8c6-47f7-8809-c7cba7c7e97f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:38.905374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4df3c-b8c6-47f7-8809-c7cba7c7e97f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:38.940713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:38.947652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f5843-13f2-42c6-a874-65105ce379ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:38.948389 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5843-13f2-42c6-a874-65105ce379ea None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:38.948673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5843-13f2-42c6-a874-65105ce379ea None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:38.948947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5843-13f2-42c6-a874-65105ce379ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:38.949868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f5843-13f2-42c6-a874-65105ce379ea None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=27f771f20b734b679d5553d29a8a3884) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:38.950154 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f5843-13f2-42c6-a874-65105ce379ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:38.951608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f5843-13f2-42c6-a874-65105ce379ea None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:38.963043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 607/1219] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:51:38 2022] GET /identity/v3/role_assignments?user.id=27f771f20b734b679d5553d29a8a3884&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:51:40.910731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae65783-23c8-4940-abef-1ace234bf5ea None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:40.911400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae65783-23c8-4940-abef-1ace234bf5ea None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:40.911400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae65783-23c8-4940-abef-1ace234bf5ea None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:40.912619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 613/1220] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:51:40.918755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cdca4-7c7a-4886-a527-716df731a4dd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:40.919274 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cdca4-7c7a-4886-a527-716df731a4dd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:40.919453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cdca4-7c7a-4886-a527-716df731a4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:40.943401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ff9cdca4-7c7a-4886-a527-716df731a4dd None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:51:40.944599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ff9cdca4-7c7a-4886-a527-716df731a4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:51:41.032286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 608/1221] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:51:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:51:41.315859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae65783-23c8-4940-abef-1ace234bf5ea None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:41.427416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae65783-23c8-4940-abef-1ace234bf5ea None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:41.463889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea231a3e-3c2f-4de6-b263-637b86fb5832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:41.464513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea231a3e-3c2f-4de6-b263-637b86fb5832 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:41.464789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea231a3e-3c2f-4de6-b263-637b86fb5832 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:41.465216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea231a3e-3c2f-4de6-b263-637b86fb5832 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:41.475578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea231a3e-3c2f-4de6-b263-637b86fb5832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ea231a3e-3c2f-4de6-b263-637b86fb5832 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:51:41.480900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:41.482023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:41.483116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 614/1222] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:51:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:41.488178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cdca4-7c7a-4886-a527-716df731a4dd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:41.596723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cdca4-7c7a-4886-a527-716df731a4dd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:41.620132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:41.630972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94d18f-4e0f-4202-b7ab-71cdbe6f6c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:41.631687 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d18f-4e0f-4202-b7ab-71cdbe6f6c9a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:41.631991 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d18f-4e0f-4202-b7ab-71cdbe6f6c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:41.632262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d18f-4e0f-4202-b7ab-71cdbe6f6c9a None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:41.632915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94d18f-4e0f-4202-b7ab-71cdbe6f6c9a None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:41.633213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94d18f-4e0f-4202-b7ab-71cdbe6f6c9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:41.634659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94d18f-4e0f-4202-b7ab-71cdbe6f6c9a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:41.643358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 609/1223] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:51:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:41.649261 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea231a3e-3c2f-4de6-b263-637b86fb5832 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:41.701682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:41.759696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea231a3e-3c2f-4de6-b263-637b86fb5832 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:41.799355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde8d91-84ba-41a4-9990-9257ef07acdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:41.799992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde8d91-84ba-41a4-9990-9257ef07acdd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:41.800274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde8d91-84ba-41a4-9990-9257ef07acdd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:41.800554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde8d91-84ba-41a4-9990-9257ef07acdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:41.811211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde8d91-84ba-41a4-9990-9257ef07acdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:41.812650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde8d91-84ba-41a4-9990-9257ef07acdd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2fde8d91-84ba-41a4-9990-9257ef07acdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:41.829255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:41.830854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:41.832095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:41.832095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:41.832095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:41.832095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 615/1224] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:41.837819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94d18f-4e0f-4202-b7ab-71cdbe6f6c9a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:41.948466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94d18f-4e0f-4202-b7ab-71cdbe6f6c9a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:41.982772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938fd2-3ab9-42fe-96e3-9680c40c3449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:41.983444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938fd2-3ab9-42fe-96e3-9680c40c3449 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:41.983725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938fd2-3ab9-42fe-96e3-9680c40c3449 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:41.984002 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938fd2-3ab9-42fe-96e3-9680c40c3449 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:41.984668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81938fd2-3ab9-42fe-96e3-9680c40c3449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:41.984972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81938fd2-3ab9-42fe-96e3-9680c40c3449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:41.986403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81938fd2-3ab9-42fe-96e3-9680c40c3449 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:42.004136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 610/1225] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:51:42.005337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde8d91-84ba-41a4-9990-9257ef07acdd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:42.112610 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:42.119601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde8d91-84ba-41a4-9990-9257ef07acdd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:42.159029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840fb90-5f45-4cb2-8bfb-26ba58c506d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:42.159029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840fb90-5f45-4cb2-8bfb-26ba58c506d4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:42.159340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840fb90-5f45-4cb2-8bfb-26ba58c506d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:42.159563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840fb90-5f45-4cb2-8bfb-26ba58c506d4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:42.167691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840fb90-5f45-4cb2-8bfb-26ba58c506d4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:42.169114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840fb90-5f45-4cb2-8bfb-26ba58c506d4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7840fb90-5f45-4cb2-8bfb-26ba58c506d4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:51:42.180890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:42.182273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:42.183691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:42.183691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:42.183691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:51:42.183691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:51:42.183691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:51:42.183691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:51:42.183691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:51:42.183691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:51:42.183691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:51:42.183691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:42.183691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 616/1226] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:42.188691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938fd2-3ab9-42fe-96e3-9680c40c3449 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:42.247163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:42.306152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938fd2-3ab9-42fe-96e3-9680c40c3449 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:42.351024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86a049-0857-4226-8191-43eccffc3241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:42.351909 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a049-0857-4226-8191-43eccffc3241 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:42.352294 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a049-0857-4226-8191-43eccffc3241 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:42.352566 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a049-0857-4226-8191-43eccffc3241 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:42.353303 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86a049-0857-4226-8191-43eccffc3241 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:42.353607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86a049-0857-4226-8191-43eccffc3241 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:42.356066 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86a049-0857-4226-8191-43eccffc3241 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:42.380536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 611/1227] 10.222.0.86 () {56 vars in 1143 bytes} [Mon Aug 29 18:51:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:42.387040 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840fb90-5f45-4cb2-8bfb-26ba58c506d4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:42.500862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840fb90-5f45-4cb2-8bfb-26ba58c506d4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:42.540467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a5bd7-ffbc-4696-b475-0fe151d210ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:42.541170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a5bd7-ffbc-4696-b475-0fe151d210ab None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:42.541968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a5bd7-ffbc-4696-b475-0fe151d210ab None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:42.542417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a5bd7-ffbc-4696-b475-0fe151d210ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:42.554308 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a5bd7-ffbc-4696-b475-0fe151d210ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:42.555883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a5bd7-ffbc-4696-b475-0fe151d210ab None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-611a5bd7-ffbc-4696-b475-0fe151d210ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:42.573722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:42.579801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:42.580936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:42.580936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:42.580936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:42.580936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 617/1228] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:51:42.581213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86a049-0857-4226-8191-43eccffc3241 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:42.695540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86a049-0857-4226-8191-43eccffc3241 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:42.698413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:42.731845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376d823-8106-4a9b-8b4a-99704d57a87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:42.732443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376d823-8106-4a9b-8b4a-99704d57a87d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:42.732727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376d823-8106-4a9b-8b4a-99704d57a87d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:42.733041 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376d823-8106-4a9b-8b4a-99704d57a87d None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:42.733625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376d823-8106-4a9b-8b4a-99704d57a87d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:42.733923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376d823-8106-4a9b-8b4a-99704d57a87d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:42.735334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376d823-8106-4a9b-8b4a-99704d57a87d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:42.748589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 612/1229] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:42.754687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a5bd7-ffbc-4696-b475-0fe151d210ab None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:42.765191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:42.870006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a5bd7-ffbc-4696-b475-0fe151d210ab None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:42.909528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6559c8e-6988-452c-a248-73fedc09c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:42.910155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6559c8e-6988-452c-a248-73fedc09c8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:42.910431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6559c8e-6988-452c-a248-73fedc09c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:42.910703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6559c8e-6988-452c-a248-73fedc09c8e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:42.918884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6559c8e-6988-452c-a248-73fedc09c8e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:42.920519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6559c8e-6988-452c-a248-73fedc09c8e5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c6559c8e-6988-452c-a248-73fedc09c8e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:51:42.933365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:42.934661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:42.935603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:51:42.935603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:51:42.935603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:51:42.935603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:51:42.935603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:51:42.935603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:51:42.935603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:51:42.935603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:42.935603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 618/1230] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:51:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:42.940724 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376d823-8106-4a9b-8b4a-99704d57a87d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:43.049749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376d823-8106-4a9b-8b4a-99704d57a87d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:43.090886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f1f28-ad8e-470b-a4f7-e8fe965c139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:43.091609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f1f28-ad8e-470b-a4f7-e8fe965c139b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:43.091958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f1f28-ad8e-470b-a4f7-e8fe965c139b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:43.092291 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f1f28-ad8e-470b-a4f7-e8fe965c139b None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:43.093057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f1f28-ad8e-470b-a4f7-e8fe965c139b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:43.093413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f1f28-ad8e-470b-a4f7-e8fe965c139b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:43.095045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f1f28-ad8e-470b-a4f7-e8fe965c139b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:43.109456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 613/1231] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:51:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:51:43.115530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6559c8e-6988-452c-a248-73fedc09c8e5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:43.186341 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:43.226408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6559c8e-6988-452c-a248-73fedc09c8e5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:43.265617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a4425-4961-4e7a-9a3f-c51c439b812f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:43.266275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a4425-4961-4e7a-9a3f-c51c439b812f None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:43.266573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a4425-4961-4e7a-9a3f-c51c439b812f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:43.266823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a4425-4961-4e7a-9a3f-c51c439b812f None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/27f771f20b734b679d5553d29a8a3884/roles/000d8f9e15c142349b2f85e8ca441e96` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:43.285651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a4425-4961-4e7a-9a3f-c51c439b812f None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=27f771f20b734b679d5553d29a8a3884, role_id=000d8f9e15c142349b2f85e8ca441e96)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:43.287261 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a4425-4961-4e7a-9a3f-c51c439b812f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:43.302413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 619/1232] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:51:43 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/27f771f20b734b679d5553d29a8a3884/roles/000d8f9e15c142349b2f85e8ca441e96 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:51:45.320875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896f620-1c60-499c-bad8-ae06285c275b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:45.321659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896f620-1c60-499c-bad8-ae06285c275b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:45.321761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896f620-1c60-499c-bad8-ae06285c275b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:45.323105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 614/1233] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:51:45.328543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccde64-709a-455c-bc13-607754e0686f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:45.328921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccde64-709a-455c-bc13-607754e0686f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:45.329097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccde64-709a-455c-bc13-607754e0686f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:45.349074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0dccde64-709a-455c-bc13-607754e0686f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:51:45.350365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0dccde64-709a-455c-bc13-607754e0686f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:51:45.424848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 620/1234] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:51:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:51:45.702199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896f620-1c60-499c-bad8-ae06285c275b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:45.733006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:45.817201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896f620-1c60-499c-bad8-ae06285c275b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:45.861863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a779e1-850d-4627-a448-c438c7f350f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:45.862656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a779e1-850d-4627-a448-c438c7f350f4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:45.863021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a779e1-850d-4627-a448-c438c7f350f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:45.863364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a779e1-850d-4627-a448-c438c7f350f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:45.876057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a779e1-850d-4627-a448-c438c7f350f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e1a779e1-850d-4627-a448-c438c7f350f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:51:45.881520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:45.882528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:45.889326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 615/1235] 10.222.0.86 () {56 vars in 1102 bytes} [Mon Aug 29 18:51:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:45.890034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccde64-709a-455c-bc13-607754e0686f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:45.947124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:46.003500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccde64-709a-455c-bc13-607754e0686f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:46.044302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fa265-adb7-48c6-8066-64891caa93c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:46.044915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fa265-adb7-48c6-8066-64891caa93c7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:46.045195 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fa265-adb7-48c6-8066-64891caa93c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:46.045474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fa265-adb7-48c6-8066-64891caa93c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:46.046287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75fa265-adb7-48c6-8066-64891caa93c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:46.046835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75fa265-adb7-48c6-8066-64891caa93c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:46.049286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75fa265-adb7-48c6-8066-64891caa93c7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:46.060048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 621/1236] 10.222.0.86 () {56 vars in 1103 bytes} [Mon Aug 29 18:51:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:51:46.066009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a779e1-850d-4627-a448-c438c7f350f4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:46.171241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:46.191482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a779e1-850d-4627-a448-c438c7f350f4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:46.231180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520f769-4a8b-4b86-9f7c-024f79293a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:46.231737 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520f769-4a8b-4b86-9f7c-024f79293a53 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:46.232015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520f769-4a8b-4b86-9f7c-024f79293a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:46.232285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520f769-4a8b-4b86-9f7c-024f79293a53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:46.242762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520f769-4a8b-4b86-9f7c-024f79293a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:46.244247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520f769-4a8b-4b86-9f7c-024f79293a53 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-3520f769-4a8b-4b86-9f7c-024f79293a53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:46.261277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:46.262645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:46.263779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:46.263779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:46.263779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:46.264082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 616/1237] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:46.270341 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fa265-adb7-48c6-8066-64891caa93c7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:46.379838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fa265-adb7-48c6-8066-64891caa93c7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:46.390758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:46.420638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4cbba-389d-462f-833a-e43a44170a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:46.421247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4cbba-389d-462f-833a-e43a44170a4e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:46.421528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4cbba-389d-462f-833a-e43a44170a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:46.421816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4cbba-389d-462f-833a-e43a44170a4e None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:46.422438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4cbba-389d-462f-833a-e43a44170a4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:46.422787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4cbba-389d-462f-833a-e43a44170a4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:46.424360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4cbba-389d-462f-833a-e43a44170a4e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:46.438700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 622/1238] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:46.444248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520f769-4a8b-4b86-9f7c-024f79293a53 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:46.554725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520f769-4a8b-4b86-9f7c-024f79293a53 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:46.592797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26853e48-10b9-490f-b393-3b1b152c1b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:46.593452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26853e48-10b9-490f-b393-3b1b152c1b07 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:46.593772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26853e48-10b9-490f-b393-3b1b152c1b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:46.594053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26853e48-10b9-490f-b393-3b1b152c1b07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:46.597293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:46.603559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26853e48-10b9-490f-b393-3b1b152c1b07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:46.605826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26853e48-10b9-490f-b393-3b1b152c1b07 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-26853e48-10b9-490f-b393-3b1b152c1b07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:51:46.617530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:46.618776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:46.620091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:46.620091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:46.620091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:51:46.620091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:51:46.620091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:51:46.620091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:51:46.620091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:51:46.620091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:51:46.620091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:51:46.620091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:46.620091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 617/1239] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:46.625088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4cbba-389d-462f-833a-e43a44170a4e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:46.740483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4cbba-389d-462f-833a-e43a44170a4e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:46.783512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ef80e-f317-47dd-9860-ca1600a833df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:46.784383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ef80e-f317-47dd-9860-ca1600a833df None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:46.784816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ef80e-f317-47dd-9860-ca1600a833df None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:46.785256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ef80e-f317-47dd-9860-ca1600a833df None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:46.786398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ef80e-f317-47dd-9860-ca1600a833df None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:46.786863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ef80e-f317-47dd-9860-ca1600a833df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:46.789045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ef80e-f317-47dd-9860-ca1600a833df None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:46.813867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 623/1240] 10.222.0.86 () {56 vars in 1143 bytes} [Mon Aug 29 18:51:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:46.815087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26853e48-10b9-490f-b393-3b1b152c1b07 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:46.929721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26853e48-10b9-490f-b393-3b1b152c1b07 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:46.965939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea5391-a715-41f5-94d2-f4b70954a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:46.966541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea5391-a715-41f5-94d2-f4b70954a027 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:46.966818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea5391-a715-41f5-94d2-f4b70954a027 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:46.967120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea5391-a715-41f5-94d2-f4b70954a027 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:47.037643 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:47.044802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea5391-a715-41f5-94d2-f4b70954a027 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:47.046255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea5391-a715-41f5-94d2-f4b70954a027 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:47.161191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 618/1241] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:51:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:51:47.166972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ef80e-f317-47dd-9860-ca1600a833df None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:47.197084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:47.284201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ef80e-f317-47dd-9860-ca1600a833df None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:47.322139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852d709-7dc5-45b8-8b25-6281d219d834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:47.323395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852d709-7dc5-45b8-8b25-6281d219d834 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:47.323666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852d709-7dc5-45b8-8b25-6281d219d834 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:47.323998 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852d709-7dc5-45b8-8b25-6281d219d834 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:47.335822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852d709-7dc5-45b8-8b25-6281d219d834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:47.337423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852d709-7dc5-45b8-8b25-6281d219d834 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e852d709-7dc5-45b8-8b25-6281d219d834 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:47.353295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:47.354431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:47.355604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:47.355604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:47.355604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:47.355604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 624/1242] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:47.360546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea5391-a715-41f5-94d2-f4b70954a027 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:47.467203 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:47.473568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea5391-a715-41f5-94d2-f4b70954a027 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:47.514865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37531c-b0c5-413b-987e-c3a3a7da037b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:47.515654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37531c-b0c5-413b-987e-c3a3a7da037b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:47.515944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37531c-b0c5-413b-987e-c3a3a7da037b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:47.516196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37531c-b0c5-413b-987e-c3a3a7da037b None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:47.516835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37531c-b0c5-413b-987e-c3a3a7da037b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:47.517133 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37531c-b0c5-413b-987e-c3a3a7da037b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:47.518487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37531c-b0c5-413b-987e-c3a3a7da037b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:47.532151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 619/1243] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:51:47.537736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852d709-7dc5-45b8-8b25-6281d219d834 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:47.628177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:47.650783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852d709-7dc5-45b8-8b25-6281d219d834 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:47.689944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa99601-7dc9-4fdc-be6a-6bacd0c7026d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:47.690569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa99601-7dc9-4fdc-be6a-6bacd0c7026d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:47.690846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa99601-7dc9-4fdc-be6a-6bacd0c7026d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:47.691144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa99601-7dc9-4fdc-be6a-6bacd0c7026d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:47.699076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa99601-7dc9-4fdc-be6a-6bacd0c7026d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:47.700498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa99601-7dc9-4fdc-be6a-6bacd0c7026d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7aa99601-7dc9-4fdc-be6a-6bacd0c7026d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:51:47.713746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:47.714823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:47.715805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:51:47.715805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:51:47.715805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:51:47.715805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:51:47.715805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:51:47.715805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:51:47.715805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:51:47.715805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:47.720443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 625/1244] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:51:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:47.721417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37531c-b0c5-413b-987e-c3a3a7da037b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:47.832731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37531c-b0c5-413b-987e-c3a3a7da037b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:47.873792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d353b-834b-4c7c-b614-c7e2e6082f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:47.874421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d353b-834b-4c7c-b614-c7e2e6082f9a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:47.874780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d353b-834b-4c7c-b614-c7e2e6082f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:47.875104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d353b-834b-4c7c-b614-c7e2e6082f9a None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:47.875867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d353b-834b-4c7c-b614-c7e2e6082f9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:47.876296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d353b-834b-4c7c-b614-c7e2e6082f9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:47.878109 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d353b-834b-4c7c-b614-c7e2e6082f9a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:47.892272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 620/1245] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:51:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:47.900617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa99601-7dc9-4fdc-be6a-6bacd0c7026d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:48.014462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa99601-7dc9-4fdc-be6a-6bacd0c7026d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:48.052359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66806f11-3dcd-4628-afcb-f5362fdee96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:48.052963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66806f11-3dcd-4628-afcb-f5362fdee96c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:48.053244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66806f11-3dcd-4628-afcb-f5362fdee96c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:48.053508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66806f11-3dcd-4628-afcb-f5362fdee96c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:48.054221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66806f11-3dcd-4628-afcb-f5362fdee96c None admin] RBAC: Adding query filter params (role.id=000d8f9e15c142349b2f85e8ca441e96, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=27f771f20b734b679d5553d29a8a3884) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:48.054523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66806f11-3dcd-4628-afcb-f5362fdee96c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:48.056143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66806f11-3dcd-4628-afcb-f5362fdee96c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:48.064482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 626/1246] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:51:47 2022] GET /identity/v3/role_assignments?user.id=27f771f20b734b679d5553d29a8a3884&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=000d8f9e15c142349b2f85e8ca441e96 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:51:50.169882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e30926-8569-4d44-92fa-7046f8833e04 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:50.170472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e30926-8569-4d44-92fa-7046f8833e04 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:50.170472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e30926-8569-4d44-92fa-7046f8833e04 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:50.172084 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 621/1247] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:51:50.178096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367e49-a703-475b-9567-ecc6cef280cf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:50.178567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367e49-a703-475b-9567-ecc6cef280cf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:50.178696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367e49-a703-475b-9567-ecc6cef280cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:50.186335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:50.200864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0b367e49-a703-475b-9567-ecc6cef280cf None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:51:50.202250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0b367e49-a703-475b-9567-ecc6cef280cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:51:50.278260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 627/1248] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:51:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:51:50.566026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e30926-8569-4d44-92fa-7046f8833e04 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:50.595241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:50.698342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e30926-8569-4d44-92fa-7046f8833e04 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:50.738331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a373e76-1f2a-4984-b4af-b7b0e8379165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:50.739007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a373e76-1f2a-4984-b4af-b7b0e8379165 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:50.739297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a373e76-1f2a-4984-b4af-b7b0e8379165 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:50.739629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a373e76-1f2a-4984-b4af-b7b0e8379165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:50.751278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a373e76-1f2a-4984-b4af-b7b0e8379165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2a373e76-1f2a-4984-b4af-b7b0e8379165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:51:50.756542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:50.757689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:50.758796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 622/1249] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:51:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:50.764464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b367e49-a703-475b-9567-ecc6cef280cf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:50.875718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b367e49-a703-475b-9567-ecc6cef280cf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:50.910554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:50.916190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003db369-df21-46e3-b4e3-b8106b909dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:50.916842 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003db369-df21-46e3-b4e3-b8106b909dea None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:50.917124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003db369-df21-46e3-b4e3-b8106b909dea None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:50.917397 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003db369-df21-46e3-b4e3-b8106b909dea None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:50.918255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003db369-df21-46e3-b4e3-b8106b909dea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:50.918843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003db369-df21-46e3-b4e3-b8106b909dea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:50.919922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003db369-df21-46e3-b4e3-b8106b909dea None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:50.931852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 628/1250] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:51:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:50.938734 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a373e76-1f2a-4984-b4af-b7b0e8379165 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:51.048222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:51.080029 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a373e76-1f2a-4984-b4af-b7b0e8379165 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:51.122209 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8099d9a-14fc-4233-9909-625da581cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:51.122922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8099d9a-14fc-4233-9909-625da581cf45 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:51.123360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8099d9a-14fc-4233-9909-625da581cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:51.123638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8099d9a-14fc-4233-9909-625da581cf45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:51.136716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8099d9a-14fc-4233-9909-625da581cf45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:51.138084 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8099d9a-14fc-4233-9909-625da581cf45 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c8099d9a-14fc-4233-9909-625da581cf45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:51.156560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:51.157759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:51.158673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:51.158673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:51.158673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:51.158673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 623/1251] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:51.163922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003db369-df21-46e3-b4e3-b8106b909dea None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:51.283773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003db369-df21-46e3-b4e3-b8106b909dea None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:51.329308 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863afd9-8dce-4641-a895-a3d569d99699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:51.329969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863afd9-8dce-4641-a895-a3d569d99699 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:51.330249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863afd9-8dce-4641-a895-a3d569d99699 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:51.330519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863afd9-8dce-4641-a895-a3d569d99699 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:51.331196 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863afd9-8dce-4641-a895-a3d569d99699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:51.331497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863afd9-8dce-4641-a895-a3d569d99699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:51.332863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863afd9-8dce-4641-a895-a3d569d99699 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:51.348627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 629/1252] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:51:51.354300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8099d9a-14fc-4233-9909-625da581cf45 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:51.474873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8099d9a-14fc-4233-9909-625da581cf45 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:51.504723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:51.515356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f0ffe-2dff-4d24-b0ed-46a57ecd354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:51.516195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f0ffe-2dff-4d24-b0ed-46a57ecd354a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:51.516520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f0ffe-2dff-4d24-b0ed-46a57ecd354a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:51.516838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f0ffe-2dff-4d24-b0ed-46a57ecd354a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:51.526642 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f0ffe-2dff-4d24-b0ed-46a57ecd354a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:51.528357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f0ffe-2dff-4d24-b0ed-46a57ecd354a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2d1f0ffe-2dff-4d24-b0ed-46a57ecd354a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:51:51.542172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:51.543580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:51.544850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:51.544850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:51.544850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:51:51.544850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:51:51.544850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:51:51.544850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:51:51.544850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:51:51.544850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:51:51.544850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:51:51.544850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:51.544850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 624/1253] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:51.550112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863afd9-8dce-4641-a895-a3d569d99699 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:51.613414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:51.663534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863afd9-8dce-4641-a895-a3d569d99699 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:51.714413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2d975-227b-477b-a5d5-f6aedae2d5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:51.715097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2d975-227b-477b-a5d5-f6aedae2d5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:51.715375 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2d975-227b-477b-a5d5-f6aedae2d5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:51.715645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2d975-227b-477b-a5d5-f6aedae2d5f3 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:51.716378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2d975-227b-477b-a5d5-f6aedae2d5f3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:51.716673 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2d975-227b-477b-a5d5-f6aedae2d5f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:51.718493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2d975-227b-477b-a5d5-f6aedae2d5f3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:51.738888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 630/1254] 10.222.0.86 () {56 vars in 1143 bytes} [Mon Aug 29 18:51:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:51.745633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f0ffe-2dff-4d24-b0ed-46a57ecd354a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:51.860290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f0ffe-2dff-4d24-b0ed-46a57ecd354a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:51.898888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed7ec7-5eef-42c7-bd34-dc949621e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:51.899524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed7ec7-5eef-42c7-bd34-dc949621e814 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:51.899805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed7ec7-5eef-42c7-bd34-dc949621e814 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:51.900078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed7ec7-5eef-42c7-bd34-dc949621e814 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:51.961667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:51.976507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed7ec7-5eef-42c7-bd34-dc949621e814 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:51.977955 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed7ec7-5eef-42c7-bd34-dc949621e814 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:52.091202 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 625/1255] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:51:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:51:52.097901 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2d975-227b-477b-a5d5-f6aedae2d5f3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:52.215598 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2d975-227b-477b-a5d5-f6aedae2d5f3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:52.230004 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:52.259942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e4705-140e-4178-99f9-90a5189a61f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:52.261228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4705-140e-4178-99f9-90a5189a61f0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:52.261626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4705-140e-4178-99f9-90a5189a61f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:52.261930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4705-140e-4178-99f9-90a5189a61f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:52.275336 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e4705-140e-4178-99f9-90a5189a61f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:52.277172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e4705-140e-4178-99f9-90a5189a61f0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-405e4705-140e-4178-99f9-90a5189a61f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:52.297756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:52.299487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:52.300770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:52.300770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:52.300770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:52.300770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 631/1256] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:52.305375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed7ec7-5eef-42c7-bd34-dc949621e814 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:52.400785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:52.414682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed7ec7-5eef-42c7-bd34-dc949621e814 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:52.453401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2791a9-fe61-4e26-991e-9885ed3b77d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:52.454020 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2791a9-fe61-4e26-991e-9885ed3b77d3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:52.454300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2791a9-fe61-4e26-991e-9885ed3b77d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:52.454572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2791a9-fe61-4e26-991e-9885ed3b77d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:52.455250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2791a9-fe61-4e26-991e-9885ed3b77d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:52.455554 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2791a9-fe61-4e26-991e-9885ed3b77d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:52.456911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2791a9-fe61-4e26-991e-9885ed3b77d3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:52.468763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 626/1257] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:52.474602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e4705-140e-4178-99f9-90a5189a61f0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:52.584185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e4705-140e-4178-99f9-90a5189a61f0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:52.621507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76afec-c993-4171-b456-29381c25efab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:52.622133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76afec-c993-4171-b456-29381c25efab None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:52.622422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76afec-c993-4171-b456-29381c25efab None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:52.622686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76afec-c993-4171-b456-29381c25efab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:52.629466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76afec-c993-4171-b456-29381c25efab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:52.630905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76afec-c993-4171-b456-29381c25efab None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fb76afec-c993-4171-b456-29381c25efab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:51:52.643166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:52.644153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:52.645059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:51:52.645059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:51:52.645059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:51:52.645059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:51:52.645059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:51:52.645059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:51:52.645059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:51:52.645059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:52.645059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 632/1258] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:51:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:52.650251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2791a9-fe61-4e26-991e-9885ed3b77d3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:52.762314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2791a9-fe61-4e26-991e-9885ed3b77d3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:52.801640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89588151-a634-4609-a950-b2ea46be6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:52.802360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89588151-a634-4609-a950-b2ea46be6a84 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:52.802655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89588151-a634-4609-a950-b2ea46be6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:52.802929 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89588151-a634-4609-a950-b2ea46be6a84 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:52.803763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89588151-a634-4609-a950-b2ea46be6a84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:52.804171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89588151-a634-4609-a950-b2ea46be6a84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:52.805916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89588151-a634-4609-a950-b2ea46be6a84 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:52.819018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 627/1259] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:51:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:52.825720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76afec-c993-4171-b456-29381c25efab None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:52.857227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:52.940222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76afec-c993-4171-b456-29381c25efab None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:52.976658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa48d2-6f62-41db-bb06-3015f0b70d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:52.977268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa48d2-6f62-41db-bb06-3015f0b70d67 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:52.977547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa48d2-6f62-41db-bb06-3015f0b70d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:52.977819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa48d2-6f62-41db-bb06-3015f0b70d67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:52.978561 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa48d2-6f62-41db-bb06-3015f0b70d67 None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=27f771f20b734b679d5553d29a8a3884) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:52.979446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa48d2-6f62-41db-bb06-3015f0b70d67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:52.980878 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa48d2-6f62-41db-bb06-3015f0b70d67 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:52.986639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 633/1260] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:51:52 2022] GET /identity/v3/role_assignments?user.id=27f771f20b734b679d5553d29a8a3884&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:54.888474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd28f4f-be01-4c8f-8c6e-959937788130 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:54.888990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd28f4f-be01-4c8f-8c6e-959937788130 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:54.889097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd28f4f-be01-4c8f-8c6e-959937788130 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:54.890344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 628/1261] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:51:54.895433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3c435-53ac-4577-b4a5-492eefb395f3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:54.895803 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3c435-53ac-4577-b4a5-492eefb395f3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:54.895978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3c435-53ac-4577-b4a5-492eefb395f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:54.914242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-aca3c435-53ac-4577-b4a5-492eefb395f3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:51:54.915437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aca3c435-53ac-4577-b4a5-492eefb395f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:51:54.979917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 634/1262] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:51:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:51:55.255183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd28f4f-be01-4c8f-8c6e-959937788130 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:55.266189 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:55.355721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd28f4f-be01-4c8f-8c6e-959937788130 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:55.388265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527980c-57fe-43ff-b148-317f2e30d27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:55.388882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527980c-57fe-43ff-b148-317f2e30d27f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:55.389157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527980c-57fe-43ff-b148-317f2e30d27f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:55.389425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527980c-57fe-43ff-b148-317f2e30d27f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:55.399055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527980c-57fe-43ff-b148-317f2e30d27f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2527980c-57fe-43ff-b148-317f2e30d27f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:51:55.404278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:55.405485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:55.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 629/1263] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:51:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:55.411390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3c435-53ac-4577-b4a5-492eefb395f3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:55.429794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:55.510715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3c435-53ac-4577-b4a5-492eefb395f3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:55.542810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96c47a-9cf7-4f65-bf94-efb7d7964bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:55.543435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96c47a-9cf7-4f65-bf94-efb7d7964bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:55.543712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96c47a-9cf7-4f65-bf94-efb7d7964bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:55.543981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96c47a-9cf7-4f65-bf94-efb7d7964bf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:55.544629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96c47a-9cf7-4f65-bf94-efb7d7964bf4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:55.544930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96c47a-9cf7-4f65-bf94-efb7d7964bf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:55.546278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96c47a-9cf7-4f65-bf94-efb7d7964bf4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:55.553999 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 635/1264] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:51:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:55.560395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527980c-57fe-43ff-b148-317f2e30d27f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:55.626728 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:55.658256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527980c-57fe-43ff-b148-317f2e30d27f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:55.691893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518076d1-5a03-4923-a1e7-f01fa4956ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:55.692520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518076d1-5a03-4923-a1e7-f01fa4956ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:55.692798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518076d1-5a03-4923-a1e7-f01fa4956ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:55.693068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518076d1-5a03-4923-a1e7-f01fa4956ab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:55.703888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518076d1-5a03-4923-a1e7-f01fa4956ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:55.705328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518076d1-5a03-4923-a1e7-f01fa4956ab4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-518076d1-5a03-4923-a1e7-f01fa4956ab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:55.719418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:55.720499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:55.721585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:55.721585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:55.721585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:55.721585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 630/1265] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:55.726312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96c47a-9cf7-4f65-bf94-efb7d7964bf4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:55.805471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:55.825890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96c47a-9cf7-4f65-bf94-efb7d7964bf4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:55.858477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa1110-9208-4661-860d-e5c65423fff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:55.859093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa1110-9208-4661-860d-e5c65423fff1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:55.859373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa1110-9208-4661-860d-e5c65423fff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:55.859641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa1110-9208-4661-860d-e5c65423fff1 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:55.860267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa1110-9208-4661-860d-e5c65423fff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:55.860565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa1110-9208-4661-860d-e5c65423fff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:55.861910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa1110-9208-4661-860d-e5c65423fff1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:55.873247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 636/1266] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:55.878862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518076d1-5a03-4923-a1e7-f01fa4956ab4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:55.976675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518076d1-5a03-4923-a1e7-f01fa4956ab4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:55.995987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:56.010682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e946c50-dc21-4381-b192-562134d81693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:56.011415 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e946c50-dc21-4381-b192-562134d81693 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:56.011627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e946c50-dc21-4381-b192-562134d81693 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:56.011901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e946c50-dc21-4381-b192-562134d81693 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:56.019270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e946c50-dc21-4381-b192-562134d81693 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:56.020699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e946c50-dc21-4381-b192-562134d81693 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1e946c50-dc21-4381-b192-562134d81693 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:51:56.031593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:56.032710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:56.033942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:56.033942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:56.033942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:51:56.033942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:51:56.033942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:51:56.033942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:51:56.033942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:51:56.033942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:51:56.033942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:51:56.033942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:56.033942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 631/1267] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:56.038447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa1110-9208-4661-860d-e5c65423fff1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:56.147097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa1110-9208-4661-860d-e5c65423fff1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:56.178858 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44be94-7ae0-4cd0-841e-d916c0ab49cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:56.179479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44be94-7ae0-4cd0-841e-d916c0ab49cb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:56.179753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44be94-7ae0-4cd0-841e-d916c0ab49cb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:56.180018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44be94-7ae0-4cd0-841e-d916c0ab49cb None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:56.180726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44be94-7ae0-4cd0-841e-d916c0ab49cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:56.181025 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44be94-7ae0-4cd0-841e-d916c0ab49cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:56.182915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44be94-7ae0-4cd0-841e-d916c0ab49cb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:56.191258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:56.200709 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 637/1268] 10.222.0.86 () {56 vars in 1143 bytes} [Mon Aug 29 18:51:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:56.206448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e946c50-dc21-4381-b192-562134d81693 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:56.302735 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e946c50-dc21-4381-b192-562134d81693 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:56.336251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350ccd7-97a3-4374-8c3f-002b4aabadb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:56.336860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ccd7-97a3-4374-8c3f-002b4aabadb0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:56.337133 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ccd7-97a3-4374-8c3f-002b4aabadb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:56.337402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ccd7-97a3-4374-8c3f-002b4aabadb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:56.347405 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350ccd7-97a3-4374-8c3f-002b4aabadb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:56.348822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350ccd7-97a3-4374-8c3f-002b4aabadb0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1350ccd7-97a3-4374-8c3f-002b4aabadb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:56.362726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:56.363697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:56.364672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:56.364672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:56.364672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:56.364672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 632/1269] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:56.370229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44be94-7ae0-4cd0-841e-d916c0ab49cb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:56.473211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44be94-7ae0-4cd0-841e-d916c0ab49cb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:56.505961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af792-ba68-4883-b2c4-ba0ea9d9aeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:56.506547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af792-ba68-4883-b2c4-ba0ea9d9aeea None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:56.506822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af792-ba68-4883-b2c4-ba0ea9d9aeea None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:56.507116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af792-ba68-4883-b2c4-ba0ea9d9aeea None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:56.507744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16af792-ba68-4883-b2c4-ba0ea9d9aeea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:56.508039 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16af792-ba68-4883-b2c4-ba0ea9d9aeea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:56.509377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16af792-ba68-4883-b2c4-ba0ea9d9aeea None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:56.520244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 638/1270] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:56.525941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350ccd7-97a3-4374-8c3f-002b4aabadb0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:56.557741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:56.624243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350ccd7-97a3-4374-8c3f-002b4aabadb0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:56.655717 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64ada7-629e-4e27-b60b-301031f3377d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:56.656329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64ada7-629e-4e27-b60b-301031f3377d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:56.656604 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64ada7-629e-4e27-b60b-301031f3377d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:56.656869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64ada7-629e-4e27-b60b-301031f3377d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:56.663145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64ada7-629e-4e27-b60b-301031f3377d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:56.664561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64ada7-629e-4e27-b60b-301031f3377d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-bc64ada7-629e-4e27-b60b-301031f3377d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:51:56.675987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:56.677059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:56.678387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:51:56.678387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:51:56.678387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:51:56.678387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:51:56.678387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:51:56.678387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:51:56.678387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:51:56.678387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:51:56.678387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 633/1271] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:51:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:56.683474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af792-ba68-4883-b2c4-ba0ea9d9aeea None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:56.743521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:56.784996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16af792-ba68-4883-b2c4-ba0ea9d9aeea None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:56.818048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8bc05-8a4a-4f8b-a45f-9db3dc298193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:56.818651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8bc05-8a4a-4f8b-a45f-9db3dc298193 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:56.818934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8bc05-8a4a-4f8b-a45f-9db3dc298193 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:56.819237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8bc05-8a4a-4f8b-a45f-9db3dc298193 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:56.819923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8bc05-8a4a-4f8b-a45f-9db3dc298193 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:56.820230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8bc05-8a4a-4f8b-a45f-9db3dc298193 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:56.821899 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8bc05-8a4a-4f8b-a45f-9db3dc298193 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:56.833584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 639/1272] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:51:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:56.840456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64ada7-629e-4e27-b60b-301031f3377d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:56.930326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:56.939241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64ada7-629e-4e27-b60b-301031f3377d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:56.971930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dfe3a-4961-48e3-b9b7-16a0ba7eb607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:56.972584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dfe3a-4961-48e3-b9b7-16a0ba7eb607 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:56.972865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dfe3a-4961-48e3-b9b7-16a0ba7eb607 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:56.973137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dfe3a-4961-48e3-b9b7-16a0ba7eb607 None admin] PATH_INFO: `/v3/projects/0446a852a61f4d2997f9211208ae8756/users/27f771f20b734b679d5553d29a8a3884/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:56.975315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dfe3a-4961-48e3-b9b7-16a0ba7eb607 None admin] RBAC: Authorizing `identity:create_grant(project_id=0446a852a61f4d2997f9211208ae8756, user_id=27f771f20b734b679d5553d29a8a3884, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:56.976744 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dfe3a-4961-48e3-b9b7-16a0ba7eb607 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:56.989701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 634/1273] 10.222.0.86 () {58 vars in 1439 bytes} [Mon Aug 29 18:51:56 2022] PUT /identity/v3/projects/0446a852a61f4d2997f9211208ae8756/users/27f771f20b734b679d5553d29a8a3884/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:51:58.874338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78953a-59e2-4ae0-899d-3f07e962ce6e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:58.874850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78953a-59e2-4ae0-899d-3f07e962ce6e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:58.874971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78953a-59e2-4ae0-899d-3f07e962ce6e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:58.876249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 640/1274] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:51:58.881348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04d9d7-c2bb-42e0-9e27-445a99ed01ea None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:58.881740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04d9d7-c2bb-42e0-9e27-445a99ed01ea None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:58.881887 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04d9d7-c2bb-42e0-9e27-445a99ed01ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:58.900523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2f04d9d7-c2bb-42e0-9e27-445a99ed01ea None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:51:58.902282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2f04d9d7-c2bb-42e0-9e27-445a99ed01ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:51:58.966411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 635/1275] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:51:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:51:59.233987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78953a-59e2-4ae0-899d-3f07e962ce6e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:59.333541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78953a-59e2-4ae0-899d-3f07e962ce6e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:59.350053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:59.366649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967e58a-f350-4974-b91c-29ca2e4c3b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:59.367293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e58a-f350-4974-b91c-29ca2e4c3b40 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:59.367569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e58a-f350-4974-b91c-29ca2e4c3b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:59.367836 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e58a-f350-4974-b91c-29ca2e4c3b40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:59.378020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967e58a-f350-4974-b91c-29ca2e4c3b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7967e58a-f350-4974-b91c-29ca2e4c3b40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:51:59.382717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:59.383804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:59.384855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 641/1276] 10.222.0.86 () {56 vars in 1096 bytes} [Mon Aug 29 18:51:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:59.389801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04d9d7-c2bb-42e0-9e27-445a99ed01ea None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:59.467495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:59.486103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04d9d7-c2bb-42e0-9e27-445a99ed01ea None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:59.520289 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15579d2-1dab-4a78-a27b-7a9e52d017b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:59.520890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15579d2-1dab-4a78-a27b-7a9e52d017b3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:59.521166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15579d2-1dab-4a78-a27b-7a9e52d017b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:59.521436 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15579d2-1dab-4a78-a27b-7a9e52d017b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:59.522086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15579d2-1dab-4a78-a27b-7a9e52d017b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:59.522388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15579d2-1dab-4a78-a27b-7a9e52d017b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:59.523760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15579d2-1dab-4a78-a27b-7a9e52d017b3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:59.532443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 636/1277] 10.222.0.86 () {56 vars in 1099 bytes} [Mon Aug 29 18:51:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:59.538228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967e58a-f350-4974-b91c-29ca2e4c3b40 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:59.636949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967e58a-f350-4974-b91c-29ca2e4c3b40 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:59.669872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f2333-d9d2-463b-9b2d-3981e68a5437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:59.670492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f2333-d9d2-463b-9b2d-3981e68a5437 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:59.670772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f2333-d9d2-463b-9b2d-3981e68a5437 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:59.671069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f2333-d9d2-463b-9b2d-3981e68a5437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:59.680912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f2333-d9d2-463b-9b2d-3981e68a5437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:59.682383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f2333-d9d2-463b-9b2d-3981e68a5437 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e42f2333-d9d2-463b-9b2d-3981e68a5437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:51:59.697117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:51:59.698726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:51:59.699722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:51:59.699722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:51:59.699722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:51:59.699722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 642/1278] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:51:59.704726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15579d2-1dab-4a78-a27b-7a9e52d017b3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:59.801442 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15579d2-1dab-4a78-a27b-7a9e52d017b3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:59.834454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b9d5d-d026-41fe-8df3-fb5cf1174485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:59.835071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b9d5d-d026-41fe-8df3-fb5cf1174485 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:59.835355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b9d5d-d026-41fe-8df3-fb5cf1174485 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:59.835627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b9d5d-d026-41fe-8df3-fb5cf1174485 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:59.836256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b9d5d-d026-41fe-8df3-fb5cf1174485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:51:59.836559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b9d5d-d026-41fe-8df3-fb5cf1174485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:59.837910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b9d5d-d026-41fe-8df3-fb5cf1174485 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:51:59.849128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 637/1279] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:51:59.854940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f2333-d9d2-463b-9b2d-3981e68a5437 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:51:59.885795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:51:59.954148 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f2333-d9d2-463b-9b2d-3981e68a5437 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:51:59.985938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafdd4f-1476-4966-abd5-5c03a47d35ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:51:59.986548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafdd4f-1476-4966-abd5-5c03a47d35ed None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:51:59.986826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafdd4f-1476-4966-abd5-5c03a47d35ed None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:51:59.987126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafdd4f-1476-4966-abd5-5c03a47d35ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:51:59.994203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafdd4f-1476-4966-abd5-5c03a47d35ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:51:59.996182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafdd4f-1476-4966-abd5-5c03a47d35ed None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2bafdd4f-1476-4966-abd5-5c03a47d35ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:52:00.006980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:52:00.008091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:52:00.009291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:52:00.009291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:52:00.009291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:52:00.009291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:52:00.009291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:52:00.009291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:52:00.009291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:52:00.009291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:52:00.009291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:52:00.009291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:52:00.009291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 643/1280] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:51:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:52:00.014469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b9d5d-d026-41fe-8df3-fb5cf1174485 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:00.018227 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:00.118177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b9d5d-d026-41fe-8df3-fb5cf1174485 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:00.151964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c81cd9-a15f-4501-b725-837392e5e70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:00.152568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c81cd9-a15f-4501-b725-837392e5e70a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:00.152850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c81cd9-a15f-4501-b725-837392e5e70a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:00.153122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c81cd9-a15f-4501-b725-837392e5e70a None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:00.153843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c81cd9-a15f-4501-b725-837392e5e70a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:52:00.154151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c81cd9-a15f-4501-b725-837392e5e70a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:00.155606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c81cd9-a15f-4501-b725-837392e5e70a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:00.172542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 638/1281] 10.222.0.86 () {56 vars in 1143 bytes} [Mon Aug 29 18:52:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:52:00.178326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafdd4f-1476-4966-abd5-5c03a47d35ed None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:00.266680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:00.276318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafdd4f-1476-4966-abd5-5c03a47d35ed None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:00.309420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8dcbc-e3c6-49e5-a291-c7db1189cb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:00.310011 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8dcbc-e3c6-49e5-a291-c7db1189cb7d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:00.310286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8dcbc-e3c6-49e5-a291-c7db1189cb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:00.310555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8dcbc-e3c6-49e5-a291-c7db1189cb7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:00.374851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8dcbc-e3c6-49e5-a291-c7db1189cb7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:00.376237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8dcbc-e3c6-49e5-a291-c7db1189cb7d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:00.459712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:00.470228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 644/1282] 10.222.0.86 () {58 vars in 1282 bytes} [Mon Aug 29 18:52:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:52:00.476024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c81cd9-a15f-4501-b725-837392e5e70a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:00.535834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:00.573723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c81cd9-a15f-4501-b725-837392e5e70a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:00.605690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951492c4-2e49-4a52-85c9-51f7787a7e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:00.606301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951492c4-2e49-4a52-85c9-51f7787a7e21 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:00.606577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951492c4-2e49-4a52-85c9-51f7787a7e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:00.606845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951492c4-2e49-4a52-85c9-51f7787a7e21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:00.616363 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951492c4-2e49-4a52-85c9-51f7787a7e21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:00.617779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951492c4-2e49-4a52-85c9-51f7787a7e21 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-951492c4-2e49-4a52-85c9-51f7787a7e21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:52:00.632205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:52:00.633324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:52:00.634413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:52:00.634413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:52:00.634413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:52:00.635697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 639/1283] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:52:00.639601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8dcbc-e3c6-49e5-a291-c7db1189cb7d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:00.735163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8dcbc-e3c6-49e5-a291-c7db1189cb7d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:00.769490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57e39d-25f2-4374-ad3e-7d2c1fe52e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:00.770083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57e39d-25f2-4374-ad3e-7d2c1fe52e6e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:00.770360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57e39d-25f2-4374-ad3e-7d2c1fe52e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:00.770634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57e39d-25f2-4374-ad3e-7d2c1fe52e6e None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:00.771293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57e39d-25f2-4374-ad3e-7d2c1fe52e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:52:00.771598 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57e39d-25f2-4374-ad3e-7d2c1fe52e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:00.772947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57e39d-25f2-4374-ad3e-7d2c1fe52e6e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:00.783803 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 645/1284] 10.222.0.86 () {56 vars in 1109 bytes} [Mon Aug 29 18:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:52:00.789601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951492c4-2e49-4a52-85c9-51f7787a7e21 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:00.887537 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951492c4-2e49-4a52-85c9-51f7787a7e21 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:00.900383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:00.920705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ea90f-1aaa-4f2a-ab92-9e0d3c5d175f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:00.921318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ea90f-1aaa-4f2a-ab92-9e0d3c5d175f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:00.921593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ea90f-1aaa-4f2a-ab92-9e0d3c5d175f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:00.921862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ea90f-1aaa-4f2a-ab92-9e0d3c5d175f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:00.928253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ea90f-1aaa-4f2a-ab92-9e0d3c5d175f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:00.929666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ea90f-1aaa-4f2a-ab92-9e0d3c5d175f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ca9ea90f-1aaa-4f2a-ab92-9e0d3c5d175f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:52:00.941097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:52:00.941977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:52:00.942902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:52:00.942902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:52:00.942902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:52:00.942902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:52:00.942902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:52:00.942902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:52:00.942902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:52:00.942902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Aug 29 18:52:00.942902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 640/1285] 10.222.0.86 () {56 vars in 1111 bytes} [Mon Aug 29 18:52:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:52:00.948251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57e39d-25f2-4374-ad3e-7d2c1fe52e6e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:00.996783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:01.045497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57e39d-25f2-4374-ad3e-7d2c1fe52e6e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:01.081049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567605a2-0856-4f85-901b-5632baab861b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:01.081644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567605a2-0856-4f85-901b-5632baab861b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:01.081925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567605a2-0856-4f85-901b-5632baab861b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:01.082198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567605a2-0856-4f85-901b-5632baab861b None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:01.082870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567605a2-0856-4f85-901b-5632baab861b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:52:01.083194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567605a2-0856-4f85-901b-5632baab861b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:01.084605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567605a2-0856-4f85-901b-5632baab861b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:01.096791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 646/1286] 10.222.0.86 () {56 vars in 1148 bytes} [Mon Aug 29 18:52:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:52:01.103079 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ea90f-1aaa-4f2a-ab92-9e0d3c5d175f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:01.205694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ea90f-1aaa-4f2a-ab92-9e0d3c5d175f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:01.238862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a2f0b-ff9b-4d2b-bb17-83e61b3c46a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:01.239482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a2f0b-ff9b-4d2b-bb17-83e61b3c46a6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:01.239765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a2f0b-ff9b-4d2b-bb17-83e61b3c46a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:01.240045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a2f0b-ff9b-4d2b-bb17-83e61b3c46a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:01.240763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a2f0b-ff9b-4d2b-bb17-83e61b3c46a6 None admin] RBAC: Adding query filter params (role.id=470667397ce74e4dbf0b4605ade12c23, scope.project.id=0446a852a61f4d2997f9211208ae8756, user.id=27f771f20b734b679d5553d29a8a3884) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:52:01.241072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a2f0b-ff9b-4d2b-bb17-83e61b3c46a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:01.242494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a2f0b-ff9b-4d2b-bb17-83e61b3c46a6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:01.248336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 641/1287] 10.222.0.86 () {56 vars in 1374 bytes} [Mon Aug 29 18:52:01 2022] GET /identity/v3/role_assignments?user.id=27f771f20b734b679d5553d29a8a3884&scope.project.id=0446a852a61f4d2997f9211208ae8756&role.id=470667397ce74e4dbf0b4605ade12c23 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:52:03.184240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504876ee-2c83-4c14-a885-17174494310b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:03.184828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504876ee-2c83-4c14-a885-17174494310b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:03.184828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504876ee-2c83-4c14-a885-17174494310b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:03.186118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 647/1288] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:52:03.191343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8139c2-436b-4da5-97ac-b38d8fa4cb1b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:03.191794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8139c2-436b-4da5-97ac-b38d8fa4cb1b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:03.191900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8139c2-436b-4da5-97ac-b38d8fa4cb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:03.212699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9c8139c2-436b-4da5-97ac-b38d8fa4cb1b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:52:03.213746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9c8139c2-436b-4da5-97ac-b38d8fa4cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:52:03.242890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:03.279729 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 642/1289] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:52:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:52:03.555355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504876ee-2c83-4c14-a885-17174494310b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:03.655017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504876ee-2c83-4c14-a885-17174494310b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:03.665333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:03.693001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a64f2-0f82-4771-8a43-0ed1cf7bee99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:03.693892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a64f2-0f82-4771-8a43-0ed1cf7bee99 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:03.694286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a64f2-0f82-4771-8a43-0ed1cf7bee99 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:03.694695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a64f2-0f82-4771-8a43-0ed1cf7bee99 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:03.695631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a64f2-0f82-4771-8a43-0ed1cf7bee99 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:03.697978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a64f2-0f82-4771-8a43-0ed1cf7bee99 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fd8a64f2-0f82-4771-8a43-0ed1cf7bee99 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:52:03.711026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:52:03.712297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:52:03.713866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 648/1290] 10.222.0.86 () {56 vars in 1117 bytes} [Mon Aug 29 18:52:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:52:03.719919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8139c2-436b-4da5-97ac-b38d8fa4cb1b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:03.822192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8139c2-436b-4da5-97ac-b38d8fa4cb1b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:03.855609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11434aef-7c53-489c-9179-f0c3c8ee7001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:03.856206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11434aef-7c53-489c-9179-f0c3c8ee7001 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:03.856489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11434aef-7c53-489c-9179-f0c3c8ee7001 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:03.856764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11434aef-7c53-489c-9179-f0c3c8ee7001 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:03.857385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11434aef-7c53-489c-9179-f0c3c8ee7001 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:52:03.857689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11434aef-7c53-489c-9179-f0c3c8ee7001 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:03.859072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11434aef-7c53-489c-9179-f0c3c8ee7001 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:03.863762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 643/1291] 10.222.0.86 () {56 vars in 1116 bytes} [Mon Aug 29 18:52:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:52:03.869540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a64f2-0f82-4771-8a43-0ed1cf7bee99 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:03.966885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a64f2-0f82-4771-8a43-0ed1cf7bee99 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:03.998638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de3459-fac8-4f75-a411-5f346ecd2b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:03.999257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de3459-fac8-4f75-a411-5f346ecd2b44 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:03.999536 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de3459-fac8-4f75-a411-5f346ecd2b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:03.999806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de3459-fac8-4f75-a411-5f346ecd2b44 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:04.000425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de3459-fac8-4f75-a411-5f346ecd2b44 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:52:04.000719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de3459-fac8-4f75-a411-5f346ecd2b44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:04.002063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de3459-fac8-4f75-a411-5f346ecd2b44 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:04.006314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 649/1292] 10.222.0.86 () {56 vars in 1116 bytes} [Mon Aug 29 18:52:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:52:06.038684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ecd2a-c994-4fdc-b796-cdd6e47961a2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:06.039221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ecd2a-c994-4fdc-b796-cdd6e47961a2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:06.039342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ecd2a-c994-4fdc-b796-cdd6e47961a2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:06.040595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 644/1293] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:52:06.046439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d162d-f11c-445b-b3ab-5c42d7581876 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:06.046882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d162d-f11c-445b-b3ab-5c42d7581876 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:06.046983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d162d-f11c-445b-b3ab-5c42d7581876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:06.065208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-116d162d-f11c-445b-b3ab-5c42d7581876 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:52:06.066836 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-116d162d-f11c-445b-b3ab-5c42d7581876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:52:06.102826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:06.143876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 650/1294] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:52:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:52:06.411655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ecd2a-c994-4fdc-b796-cdd6e47961a2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:06.442968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:06.512362 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ecd2a-c994-4fdc-b796-cdd6e47961a2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:06.545477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f338076-7fbf-4c83-bc46-49aac419f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:06.546069 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f338076-7fbf-4c83-bc46-49aac419f3f9 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:06.546348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f338076-7fbf-4c83-bc46-49aac419f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:06.546618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f338076-7fbf-4c83-bc46-49aac419f3f9 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:06.547293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f338076-7fbf-4c83-bc46-49aac419f3f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:06.548656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f338076-7fbf-4c83-bc46-49aac419f3f9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:06.562643 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 645/1295] 10.222.0.86 () {60 vars in 1141 bytes} [Mon Aug 29 18:52:06 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:52:08.463729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa4a0d-391d-41be-8c3c-9f4a4fb7a4e9 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:08.464286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa4a0d-391d-41be-8c3c-9f4a4fb7a4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:08.464286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa4a0d-391d-41be-8c3c-9f4a4fb7a4e9 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:08.465602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 651/1296] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:52:08.470647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58750870-af8e-4ebf-8159-533eaeac66d4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:08.471100 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58750870-af8e-4ebf-8159-533eaeac66d4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:08.471201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58750870-af8e-4ebf-8159-533eaeac66d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:08.489570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-58750870-af8e-4ebf-8159-533eaeac66d4 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:52:08.490741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-58750870-af8e-4ebf-8159-533eaeac66d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:52:08.560657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 646/1297] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:52:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:52:08.828273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa4a0d-391d-41be-8c3c-9f4a4fb7a4e9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:08.926514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa4a0d-391d-41be-8c3c-9f4a4fb7a4e9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:08.959265 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206146b-4156-4bbb-b40b-98c275701955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:08.959879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206146b-4156-4bbb-b40b-98c275701955 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:08.960152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206146b-4156-4bbb-b40b-98c275701955 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:08.960416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206146b-4156-4bbb-b40b-98c275701955 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:08.960980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206146b-4156-4bbb-b40b-98c275701955 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:08.962332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206146b-4156-4bbb-b40b-98c275701955 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d206146b-4156-4bbb-b40b-98c275701955 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:52:08.971272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:52:08.972388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:52:08.973387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 652/1298] 10.222.0.86 () {56 vars in 1117 bytes} [Mon Aug 29 18:52:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:52:08.978322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58750870-af8e-4ebf-8159-533eaeac66d4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:08.996142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:09.077820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58750870-af8e-4ebf-8159-533eaeac66d4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:09.110823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2262769-bf4e-4037-aa21-142900a34fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:09.111391 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262769-bf4e-4037-aa21-142900a34fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:09.111697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262769-bf4e-4037-aa21-142900a34fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:09.111967 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262769-bf4e-4037-aa21-142900a34fd8 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:09.112581 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2262769-bf4e-4037-aa21-142900a34fd8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:52:09.112880 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2262769-bf4e-4037-aa21-142900a34fd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:09.114228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2262769-bf4e-4037-aa21-142900a34fd8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:09.119664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 647/1299] 10.222.0.86 () {56 vars in 1116 bytes} [Mon Aug 29 18:52:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:52:09.125752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206146b-4156-4bbb-b40b-98c275701955 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:09.156590 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:09.222045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206146b-4156-4bbb-b40b-98c275701955 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:09.253950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0d320-1b82-4d7c-9aea-c10415ac274c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:09.254533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0d320-1b82-4d7c-9aea-c10415ac274c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:09.254807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0d320-1b82-4d7c-9aea-c10415ac274c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:09.255098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0d320-1b82-4d7c-9aea-c10415ac274c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:09.255782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0d320-1b82-4d7c-9aea-c10415ac274c None admin] RBAC: Adding query filter params (interface=public, service_id=0484b7fc595743d685150104f70ba2d1, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:52:09.256083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0d320-1b82-4d7c-9aea-c10415ac274c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:09.257417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0d320-1b82-4d7c-9aea-c10415ac274c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:09.262940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 653/1300] 10.222.0.86 () {56 vars in 1251 bytes} [Mon Aug 29 18:52:09 2022] GET /identity/v3/endpoints?service_id=0484b7fc595743d685150104f70ba2d1&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:52:09.268678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2262769-bf4e-4037-aa21-142900a34fd8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:09.350774 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:09.368796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2262769-bf4e-4037-aa21-142900a34fd8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:09.403190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee369be7-a135-4695-869c-4fbe595e65cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:09.403792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369be7-a135-4695-869c-4fbe595e65cb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:09.404073 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369be7-a135-4695-869c-4fbe595e65cb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:09.404342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369be7-a135-4695-869c-4fbe595e65cb None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:09.404933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee369be7-a135-4695-869c-4fbe595e65cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:09.406280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee369be7-a135-4695-869c-4fbe595e65cb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:09.412942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 648/1301] 10.222.0.86 () {56 vars in 1087 bytes} [Mon Aug 29 18:52:09 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:52:11.303314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddf34d-9b16-4ce4-a906-ba6a45897822 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:11.303866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddf34d-9b16-4ce4-a906-ba6a45897822 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:11.303866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddf34d-9b16-4ce4-a906-ba6a45897822 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:11.305174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 654/1302] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:52:11.311872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0475d-62f8-4753-b34a-b4c2aa7fb9c0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:11.312343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0475d-62f8-4753-b34a-b4c2aa7fb9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:11.312442 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0475d-62f8-4753-b34a-b4c2aa7fb9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:11.331769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-57c0475d-62f8-4753-b34a-b4c2aa7fb9c0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:52:11.332950 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-57c0475d-62f8-4753-b34a-b4c2aa7fb9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:52:11.397202 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 649/1303] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:52:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:52:11.672904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddf34d-9b16-4ce4-a906-ba6a45897822 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:11.768520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:11.775723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddf34d-9b16-4ce4-a906-ba6a45897822 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:11.809795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccee5d-924f-42fd-912e-ca8c0700f59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:11.810419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccee5d-924f-42fd-912e-ca8c0700f59f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:11.810694 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccee5d-924f-42fd-912e-ca8c0700f59f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:11.810958 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccee5d-924f-42fd-912e-ca8c0700f59f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:11.811557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccee5d-924f-42fd-912e-ca8c0700f59f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:11.812900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccee5d-924f-42fd-912e-ca8c0700f59f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-83ccee5d-924f-42fd-912e-ca8c0700f59f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:52:11.821457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:52:11.822880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Aug 29 18:52:11.823784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 655/1304] 10.222.0.86 () {56 vars in 1117 bytes} [Mon Aug 29 18:52:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:52:11.828992 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0475d-62f8-4753-b34a-b4c2aa7fb9c0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:11.904157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:11.926918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0475d-62f8-4753-b34a-b4c2aa7fb9c0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:11.959890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51dcaf-fb4b-4a6f-bf36-9f66553cdc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:11.960487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51dcaf-fb4b-4a6f-bf36-9f66553cdc35 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:11.960764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51dcaf-fb4b-4a6f-bf36-9f66553cdc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:11.961033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51dcaf-fb4b-4a6f-bf36-9f66553cdc35 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:11.961647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51dcaf-fb4b-4a6f-bf36-9f66553cdc35 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:52:11.961948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51dcaf-fb4b-4a6f-bf36-9f66553cdc35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:11.963347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51dcaf-fb4b-4a6f-bf36-9f66553cdc35 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:11.968285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 650/1305] 10.222.0.86 () {56 vars in 1116 bytes} [Mon Aug 29 18:52:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:52:11.973942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccee5d-924f-42fd-912e-ca8c0700f59f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:52:12.073467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccee5d-924f-42fd-912e-ca8c0700f59f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:52:12.114232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bfe330-043a-4059-976e-582992d8500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:52:12.114895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfe330-043a-4059-976e-582992d8500b None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:52:12.115264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfe330-043a-4059-976e-582992d8500b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:52:12.115578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfe330-043a-4059-976e-582992d8500b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:52:12.116479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bfe330-043a-4059-976e-582992d8500b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:52:12.118007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bfe330-043a-4059-976e-582992d8500b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:52:12.139841 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 656/1306] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 18:52:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:52:21.965888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=30 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:52:22.135485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:16.198511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f11c5-3e35-4c76-aa0f-97407a27ff51 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:16.198511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f11c5-3e35-4c76-aa0f-97407a27ff51 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:16.198511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f11c5-3e35-4c76-aa0f-97407a27ff51 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:16.205451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2387c-49e9-4797-8eee-424d2d3a9e7a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:16.205848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2387c-49e9-4797-8eee-424d2d3a9e7a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:16.206132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2387c-49e9-4797-8eee-424d2d3a9e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:16.210695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 651/1307] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:16.409748 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-74c2387c-49e9-4797-8eee-424d2d3a9e7a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:16.413844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-74c2387c-49e9-4797-8eee-424d2d3a9e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:16.545058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 657/1308] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:16.841844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32654070-77c0-46ee-8ad1-06d0993f6254 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:16.842127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32654070-77c0-46ee-8ad1-06d0993f6254 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:16.842656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32654070-77c0-46ee-8ad1-06d0993f6254 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:16.844237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 652/1309] 10.222.0.86 () {54 vars in 965 bytes} [Mon Aug 29 18:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:16.850662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aac299-446e-44fc-a665-b06ac46e703b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:16.851175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aac299-446e-44fc-a665-b06ac46e703b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:16.851555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aac299-446e-44fc-a665-b06ac46e703b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:16.946668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-17aac299-446e-44fc-a665-b06ac46e703b None None] MFA Rules not processed for user `5ec64944b34d4ef1bd666bb1b7e216da`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:16.947863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-17aac299-446e-44fc-a665-b06ac46e703b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:16.971602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:17.033843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 658/1310] 10.222.0.86 () {58 vars in 1062 bytes} [Mon Aug 29 18:53:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:17.042874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32654070-77c0-46ee-8ad1-06d0993f6254 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:17.044825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-32654070-77c0-46ee-8ad1-06d0993f6254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:17.170145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32654070-77c0-46ee-8ad1-06d0993f6254 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:17.249869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b67385-6a5c-44c8-9a32-cc4b0d213a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:17.250509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b67385-6a5c-44c8-9a32-cc4b0d213a40 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:17.250803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b67385-6a5c-44c8-9a32-cc4b0d213a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:17.250976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b67385-6a5c-44c8-9a32-cc4b0d213a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:17.252696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-25b67385-6a5c-44c8-9a32-cc4b0d213a40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:17.310464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:17.370933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b67385-6a5c-44c8-9a32-cc4b0d213a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:17.372409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b67385-6a5c-44c8-9a32-cc4b0d213a40 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:17.393194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 653/1311] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:19.611861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da3231-642a-4b6a-a0d3-34e1724911bb None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.612320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da3231-642a-4b6a-a0d3-34e1724911bb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.612432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da3231-642a-4b6a-a0d3-34e1724911bb None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.613741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 659/1312] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:19.619155 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173416-0213-4609-9119-caec9d9accad None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.619540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173416-0213-4609-9119-caec9d9accad None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.619661 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173416-0213-4609-9119-caec9d9accad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.703742 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-72173416-0213-4609-9119-caec9d9accad None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:19.704987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-72173416-0213-4609-9119-caec9d9accad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:19.801333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 654/1313] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:53:20.080722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da3231-642a-4b6a-a0d3-34e1724911bb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:20.082253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-14da3231-642a-4b6a-a0d3-34e1724911bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:20.183388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da3231-642a-4b6a-a0d3-34e1724911bb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:20.252329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:20.254935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfca22-e38b-4a67-aecb-ff8fd62242d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:20.255579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfca22-e38b-4a67-aecb-ff8fd62242d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:20.256028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfca22-e38b-4a67-aecb-ff8fd62242d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:20.256340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfca22-e38b-4a67-aecb-ff8fd62242d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:20.258249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-73dfca22-e38b-4a67-aecb-ff8fd62242d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:20.365279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfca22-e38b-4a67-aecb-ff8fd62242d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:20.366798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfca22-e38b-4a67-aecb-ff8fd62242d4 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:20.386030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 660/1314] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:22.639881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107b915-d626-4d92-9602-610c4fb4fc60 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:22.640446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107b915-d626-4d92-9602-610c4fb4fc60 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:22.640446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107b915-d626-4d92-9602-610c4fb4fc60 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:22.641766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 655/1315] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:22.646820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a705d-b50d-46e7-8a81-fbff0b8e9ffe None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:22.647291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a705d-b50d-46e7-8a81-fbff0b8e9ffe None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:22.647410 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a705d-b50d-46e7-8a81-fbff0b8e9ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:22.720758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-749a705d-b50d-46e7-8a81-fbff0b8e9ffe None None] MFA Rules not processed for user `216c0adc13b34a498d52b72c7b75c72b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:22.721959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-749a705d-b50d-46e7-8a81-fbff0b8e9ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:22.776511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:22.820555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 661/1316] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:23.114807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107b915-d626-4d92-9602-610c4fb4fc60 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:23.116430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9107b915-d626-4d92-9602-610c4fb4fc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:23.126811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:23.219791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107b915-d626-4d92-9602-610c4fb4fc60 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:23.291928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc737e9-7b14-469c-a9e1-6a7bd7d20459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:23.292531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc737e9-7b14-469c-a9e1-6a7bd7d20459 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:23.292811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc737e9-7b14-469c-a9e1-6a7bd7d20459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:23.293109 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc737e9-7b14-469c-a9e1-6a7bd7d20459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:23.294713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8dc737e9-7b14-469c-a9e1-6a7bd7d20459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:23.355313 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:23.404676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc737e9-7b14-469c-a9e1-6a7bd7d20459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:23.406040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc737e9-7b14-469c-a9e1-6a7bd7d20459 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:23.425122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 656/1317] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:23 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:25.887680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508c973-3ce9-4f33-a301-611afe73298d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:25.888183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508c973-3ce9-4f33-a301-611afe73298d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:25.888183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508c973-3ce9-4f33-a301-611afe73298d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:25.889497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 662/1318] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:25.894538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a60da-51a3-421d-ba96-6a996b93da79 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:25.894916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a60da-51a3-421d-ba96-6a996b93da79 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:25.895128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a60da-51a3-421d-ba96-6a996b93da79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:25.959865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d21a60da-51a3-421d-ba96-6a996b93da79 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:25.961084 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d21a60da-51a3-421d-ba96-6a996b93da79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:26.056375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 657/1319] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:26.342544 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508c973-3ce9-4f33-a301-611afe73298d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:26.344118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0508c973-3ce9-4f33-a301-611afe73298d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:26.448125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508c973-3ce9-4f33-a301-611afe73298d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:26.520117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d78e3-2d2f-4c1a-a636-37e5bbd4cfed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:26.520716 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d78e3-2d2f-4c1a-a636-37e5bbd4cfed service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:26.520991 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d78e3-2d2f-4c1a-a636-37e5bbd4cfed service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:26.521260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d78e3-2d2f-4c1a-a636-37e5bbd4cfed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:26.522836 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cc1d78e3-2d2f-4c1a-a636-37e5bbd4cfed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:26.530944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:26.631415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d78e3-2d2f-4c1a-a636-37e5bbd4cfed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:26.632747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d78e3-2d2f-4c1a-a636-37e5bbd4cfed service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:26.651160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 663/1320] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:29.228444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efc5b4-7620-43b6-ab4b-f287cc85f644 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.229164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efc5b4-7620-43b6-ab4b-f287cc85f644 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.229164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efc5b4-7620-43b6-ab4b-f287cc85f644 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.230323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 658/1321] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:29.235560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3ffac-c633-490d-89bb-e0e0807e5f08 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.236021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3ffac-c633-490d-89bb-e0e0807e5f08 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.236200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3ffac-c633-490d-89bb-e0e0807e5f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.301709 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a2e3ffac-c633-490d-89bb-e0e0807e5f08 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:29.302907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a2e3ffac-c633-490d-89bb-e0e0807e5f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:29.361633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:29.402548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 664/1322] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:29.701122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2e2e-0fd9-4964-b70e-9430a1358355 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.701404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2e2e-0fd9-4964-b70e-9430a1358355 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.701682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2e2e-0fd9-4964-b70e-9430a1358355 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.703191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 659/1323] 10.222.0.86 () {54 vars in 965 bytes} [Mon Aug 29 18:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:29.708868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c24903-059e-47f9-b661-cb811342ce47 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.709230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c24903-059e-47f9-b661-cb811342ce47 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.709570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c24903-059e-47f9-b661-cb811342ce47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.790824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f7c24903-059e-47f9-b661-cb811342ce47 None None] MFA Rules not processed for user `5ec64944b34d4ef1bd666bb1b7e216da`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:29.792051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f7c24903-059e-47f9-b661-cb811342ce47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:29.867609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 665/1324] 10.222.0.86 () {58 vars in 1062 bytes} [Mon Aug 29 18:53:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:53:29.876510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e2e2e-0fd9-4964-b70e-9430a1358355 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:29.878058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-606e2e2e-0fd9-4964-b70e-9430a1358355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:29.901543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:29.985866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e2e2e-0fd9-4964-b70e-9430a1358355 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:30.062876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a7b19-20fa-455f-878f-e659573f088a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:30.063512 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a7b19-20fa-455f-878f-e659573f088a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.063793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a7b19-20fa-455f-878f-e659573f088a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.064111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a7b19-20fa-455f-878f-e659573f088a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.065705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3f4a7b19-20fa-455f-878f-e659573f088a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:30.139353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:30.179373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a7b19-20fa-455f-878f-e659573f088a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:30.180862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a7b19-20fa-455f-878f-e659573f088a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:30.200215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 660/1325] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:32.392773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7017577-0e1b-4cc3-80bc-55f5eae125a8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:32.393310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7017577-0e1b-4cc3-80bc-55f5eae125a8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:32.393310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7017577-0e1b-4cc3-80bc-55f5eae125a8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:32.394580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 666/1326] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:32.399751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d3463-d736-4ee7-a699-c662a64dc0b6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:32.400176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d3463-d736-4ee7-a699-c662a64dc0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:32.400462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d3463-d736-4ee7-a699-c662a64dc0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:32.466527 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cd5d3463-d736-4ee7-a699-c662a64dc0b6 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:32.467759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cd5d3463-d736-4ee7-a699-c662a64dc0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:32.575425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 661/1327] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:32.858809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7017577-0e1b-4cc3-80bc-55f5eae125a8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:32.860416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c7017577-0e1b-4cc3-80bc-55f5eae125a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:32.920572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:32.964525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7017577-0e1b-4cc3-80bc-55f5eae125a8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:33.036660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56597fa-48c1-483c-85b1-5e28ed257777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:33.037268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56597fa-48c1-483c-85b1-5e28ed257777 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:33.037548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56597fa-48c1-483c-85b1-5e28ed257777 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:33.037821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56597fa-48c1-483c-85b1-5e28ed257777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:33.039415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d56597fa-48c1-483c-85b1-5e28ed257777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:33.144832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:33.147306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56597fa-48c1-483c-85b1-5e28ed257777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:33.148753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56597fa-48c1-483c-85b1-5e28ed257777 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:33.168441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 667/1328] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:35.150532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c4862-e94c-4c26-ae4b-95f583293656 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:35.150532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c4862-e94c-4c26-ae4b-95f583293656 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:35.150532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c4862-e94c-4c26-ae4b-95f583293656 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:35.154388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 662/1329] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:35.160094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552419f-1cb9-4931-8250-45e977dcb6fa None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:35.160561 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552419f-1cb9-4931-8250-45e977dcb6fa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:35.160654 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552419f-1cb9-4931-8250-45e977dcb6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:35.238507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8552419f-1cb9-4931-8250-45e977dcb6fa None None] MFA Rules not processed for user `216c0adc13b34a498d52b72c7b75c72b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:35.239777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8552419f-1cb9-4931-8250-45e977dcb6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:35.338295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 668/1330] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:35.630816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c4862-e94c-4c26-ae4b-95f583293656 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:35.632470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-465c4862-e94c-4c26-ae4b-95f583293656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:35.663264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:35.753575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c4862-e94c-4c26-ae4b-95f583293656 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:35.865377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6f526-726e-4374-8341-cde1164093e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:35.866015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6f526-726e-4374-8341-cde1164093e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:35.866300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6f526-726e-4374-8341-cde1164093e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:35.866574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6f526-726e-4374-8341-cde1164093e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:35.869730 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b1f6f526-726e-4374-8341-cde1164093e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:35.964611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:35.999251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6f526-726e-4374-8341-cde1164093e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:36.000720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6f526-726e-4374-8341-cde1164093e7 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:36.020395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 663/1331] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:38.169301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7059d6e-86c8-4e15-83ac-f18c01a85f91 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.169851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7059d6e-86c8-4e15-83ac-f18c01a85f91 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.169851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7059d6e-86c8-4e15-83ac-f18c01a85f91 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.171229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 669/1332] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:38.176279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2442fc8-0167-473f-9ff2-9d70159aa571 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.176717 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2442fc8-0167-473f-9ff2-9d70159aa571 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.176838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2442fc8-0167-473f-9ff2-9d70159aa571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.247129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f2442fc8-0167-473f-9ff2-9d70159aa571 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:38.248388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f2442fc8-0167-473f-9ff2-9d70159aa571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:38.343818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 664/1333] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:38.627601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7059d6e-86c8-4e15-83ac-f18c01a85f91 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:38.629124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c7059d6e-86c8-4e15-83ac-f18c01a85f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:38.700771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:38.734959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7059d6e-86c8-4e15-83ac-f18c01a85f91 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:38.812385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b900039-eb24-434d-be05-e1a274f913cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:38.813259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b900039-eb24-434d-be05-e1a274f913cc service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.814361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b900039-eb24-434d-be05-e1a274f913cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.814361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b900039-eb24-434d-be05-e1a274f913cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.816611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7b900039-eb24-434d-be05-e1a274f913cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:38.956594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b900039-eb24-434d-be05-e1a274f913cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:38.957942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b900039-eb24-434d-be05-e1a274f913cc service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:38.967622 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:38.975873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 670/1334] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:40.930152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51f92d-eb96-4a56-afc5-4dda3b6d230c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:40.931277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51f92d-eb96-4a56-afc5-4dda3b6d230c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:40.931373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51f92d-eb96-4a56-afc5-4dda3b6d230c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:40.932689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 665/1335] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:40.937851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6395a-65b3-4bd1-98c0-7c83a9ac4d2b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:40.938241 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6395a-65b3-4bd1-98c0-7c83a9ac4d2b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:40.938339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6395a-65b3-4bd1-98c0-7c83a9ac4d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:41.003275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6ad6395a-65b3-4bd1-98c0-7c83a9ac4d2b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:41.004482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6ad6395a-65b3-4bd1-98c0-7c83a9ac4d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:41.108016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 671/1336] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:41.397069 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51f92d-eb96-4a56-afc5-4dda3b6d230c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:41.398609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cb51f92d-eb96-4a56-afc5-4dda3b6d230c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:41.438629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:41.502261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51f92d-eb96-4a56-afc5-4dda3b6d230c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:41.573375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe32d9-7326-4092-81b2-257ad199f5f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:41.574247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe32d9-7326-4092-81b2-257ad199f5f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:41.574531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe32d9-7326-4092-81b2-257ad199f5f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:41.574798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe32d9-7326-4092-81b2-257ad199f5f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:41.576408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cdbe32d9-7326-4092-81b2-257ad199f5f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:41.660849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:41.688411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe32d9-7326-4092-81b2-257ad199f5f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:41.689712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe32d9-7326-4092-81b2-257ad199f5f4 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:41.707908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 666/1337] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:43.914981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc668f-c502-4d4c-bb3e-392de082b974 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:43.915591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc668f-c502-4d4c-bb3e-392de082b974 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:43.915591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc668f-c502-4d4c-bb3e-392de082b974 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:43.916967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 672/1338] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:43.922235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fd9e0-d859-44b7-9132-430c7e4ee003 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:43.922629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fd9e0-d859-44b7-9132-430c7e4ee003 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:43.922790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fd9e0-d859-44b7-9132-430c7e4ee003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:43.987599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-989fd9e0-d859-44b7-9132-430c7e4ee003 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:43.988809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-989fd9e0-d859-44b7-9132-430c7e4ee003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:44.085820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 667/1339] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:44.378767 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc668f-c502-4d4c-bb3e-392de082b974 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:44.380441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-64dc668f-c502-4d4c-bb3e-392de082b974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:44.457056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:44.484850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc668f-c502-4d4c-bb3e-392de082b974 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:44.558240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a5478-e8a6-48f1-80ef-41e247b95dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:44.558843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a5478-e8a6-48f1-80ef-41e247b95dae service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:44.559160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a5478-e8a6-48f1-80ef-41e247b95dae service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:44.559431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a5478-e8a6-48f1-80ef-41e247b95dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:44.561004 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b58a5478-e8a6-48f1-80ef-41e247b95dae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:44.668057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a5478-e8a6-48f1-80ef-41e247b95dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:44.669398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a5478-e8a6-48f1-80ef-41e247b95dae service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:44.688333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 673/1340] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:48.506400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aba4bf-a344-4513-9ce7-b8d7c879ddc9 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:48.506860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aba4bf-a344-4513-9ce7-b8d7c879ddc9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:48.506948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aba4bf-a344-4513-9ce7-b8d7c879ddc9 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:48.508935 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 668/1341] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:48.514136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67331794-08ea-4f3e-b1d3-d7da75266c70 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:48.514566 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67331794-08ea-4f3e-b1d3-d7da75266c70 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:48.514699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67331794-08ea-4f3e-b1d3-d7da75266c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:48.520356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:48.583523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-67331794-08ea-4f3e-b1d3-d7da75266c70 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:48.584752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-67331794-08ea-4f3e-b1d3-d7da75266c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:48.685197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 674/1342] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:48.967376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aba4bf-a344-4513-9ce7-b8d7c879ddc9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:48.968932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-46aba4bf-a344-4513-9ce7-b8d7c879ddc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:49.017982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:49.081749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aba4bf-a344-4513-9ce7-b8d7c879ddc9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:49.157954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1574b-3db0-4c75-95dc-f27ddb0d7325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:49.158681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1574b-3db0-4c75-95dc-f27ddb0d7325 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:49.158965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1574b-3db0-4c75-95dc-f27ddb0d7325 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:49.159271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1574b-3db0-4c75-95dc-f27ddb0d7325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:49.160860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-afa1574b-3db0-4c75-95dc-f27ddb0d7325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:49.255057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:49.271875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa1574b-3db0-4c75-95dc-f27ddb0d7325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:49.273311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa1574b-3db0-4c75-95dc-f27ddb0d7325 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:49.292746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 669/1343] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:51.875606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fe8cf-be4b-4280-94cb-d02e0b218c3a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:51.876163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fe8cf-be4b-4280-94cb-d02e0b218c3a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:51.876163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fe8cf-be4b-4280-94cb-d02e0b218c3a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:51.877516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 675/1344] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:51.883903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da9f9e-a4db-4721-9197-edf5460017cf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:51.884455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da9f9e-a4db-4721-9197-edf5460017cf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:51.884455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da9f9e-a4db-4721-9197-edf5460017cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:51.954723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-77da9f9e-a4db-4721-9197-edf5460017cf None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:51.955990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-77da9f9e-a4db-4721-9197-edf5460017cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:52.062969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 670/1345] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:52.355788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fe8cf-be4b-4280-94cb-d02e0b218c3a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:52.358554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3e1fe8cf-be4b-4280-94cb-d02e0b218c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:52.462554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:52.485985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fe8cf-be4b-4280-94cb-d02e0b218c3a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:52.575759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c561e5-952f-447e-ac7c-d71bd6b85517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:52.576377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c561e5-952f-447e-ac7c-d71bd6b85517 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:52.576661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c561e5-952f-447e-ac7c-d71bd6b85517 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:52.577050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c561e5-952f-447e-ac7c-d71bd6b85517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:52.578661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-24c561e5-952f-447e-ac7c-d71bd6b85517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:52.704533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c561e5-952f-447e-ac7c-d71bd6b85517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:52.705972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c561e5-952f-447e-ac7c-d71bd6b85517 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:52.731554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 676/1346] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:53:56.677337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc360e10-0c91-4a0a-ab97-a8f45e8854fe None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:56.677855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc360e10-0c91-4a0a-ab97-a8f45e8854fe None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:56.677855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc360e10-0c91-4a0a-ab97-a8f45e8854fe None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:56.679293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 671/1347] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:53:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:56.685733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f2359-685d-42cf-8782-906d7632e9ed None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:56.686229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f2359-685d-42cf-8782-906d7632e9ed None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:56.686310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f2359-685d-42cf-8782-906d7632e9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:56.702772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:56.761006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-175f2359-685d-42cf-8782-906d7632e9ed None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:56.762230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-175f2359-685d-42cf-8782-906d7632e9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:56.864951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 677/1348] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:53:57.146317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc360e10-0c91-4a0a-ab97-a8f45e8854fe None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:57.148007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cc360e10-0c91-4a0a-ab97-a8f45e8854fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:57.207344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:57.259401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc360e10-0c91-4a0a-ab97-a8f45e8854fe None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:57.334671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3384e2-32de-4c44-860a-10519f560ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:57.335333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3384e2-32de-4c44-860a-10519f560ffe service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:57.335615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3384e2-32de-4c44-860a-10519f560ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:57.335889 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3384e2-32de-4c44-860a-10519f560ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:57.338112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bf3384e2-32de-4c44-860a-10519f560ffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:57.445255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:57.448907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3384e2-32de-4c44-860a-10519f560ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:57.450820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3384e2-32de-4c44-860a-10519f560ffe service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:57.469121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 672/1349] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:02.453740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eeb36-ea31-401a-938a-75c490f9beb5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:02.454192 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eeb36-ea31-401a-938a-75c490f9beb5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:02.454296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eeb36-ea31-401a-938a-75c490f9beb5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:02.455949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 678/1350] 10.222.0.86 () {54 vars in 921 bytes} [Mon Aug 29 18:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:02.463103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b18172-88b2-4c90-b276-6865392dcb78 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:02.463558 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b18172-88b2-4c90-b276-6865392dcb78 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:02.463650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b18172-88b2-4c90-b276-6865392dcb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:02.579408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-52b18172-88b2-4c90-b276-6865392dcb78 None None] MFA Rules not processed for user `27f771f20b734b679d5553d29a8a3884`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:02.580623 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-52b18172-88b2-4c90-b276-6865392dcb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:02.780644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 673/1351] 10.222.0.86 () {58 vars in 1018 bytes} [Mon Aug 29 18:54:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:06.863042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=59 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:06.927130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619480e-f281-4475-a9d7-37cf60a5fdd1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:06.927770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619480e-f281-4475-a9d7-37cf60a5fdd1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:06.928725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619480e-f281-4475-a9d7-37cf60a5fdd1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:06.930360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 679/1352] 10.222.0.86 () {54 vars in 921 bytes} [Mon Aug 29 18:54:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:06.935357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faf3e2-61d1-425d-b65d-2367e39e6946 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:06.935974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faf3e2-61d1-425d-b65d-2367e39e6946 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:06.936763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faf3e2-61d1-425d-b65d-2367e39e6946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:06.995563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:07.021193 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-39faf3e2-61d1-425d-b65d-2367e39e6946 None None] MFA Rules not processed for user `27f771f20b734b679d5553d29a8a3884`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:07.022401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-39faf3e2-61d1-425d-b65d-2367e39e6946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:07.134823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 674/1353] 10.222.0.86 () {58 vars in 1018 bytes} [Mon Aug 29 18:54:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:08.554778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2d4a9-3524-46b9-b331-6cf5e3d6fc5c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:08.555350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2d4a9-3524-46b9-b331-6cf5e3d6fc5c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:08.555441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2d4a9-3524-46b9-b331-6cf5e3d6fc5c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:08.556715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 680/1354] 10.222.0.86 () {54 vars in 921 bytes} [Mon Aug 29 18:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:08.561693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31876dd8-5644-4645-9145-a8ae7f24a1bd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:08.562101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31876dd8-5644-4645-9145-a8ae7f24a1bd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:08.562365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31876dd8-5644-4645-9145-a8ae7f24a1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:08.647663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-31876dd8-5644-4645-9145-a8ae7f24a1bd None None] MFA Rules not processed for user `27f771f20b734b679d5553d29a8a3884`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:08.649628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31876dd8-5644-4645-9145-a8ae7f24a1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:08.693906 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:08.761380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 675/1355] 10.222.0.86 () {58 vars in 1018 bytes} [Mon Aug 29 18:54:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:54:12.080516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c015f-bca6-4c45-a366-41f7c61c8525 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.081094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c015f-bca6-4c45-a366-41f7c61c8525 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.081094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c015f-bca6-4c45-a366-41f7c61c8525 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.082668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 681/1356] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:12.093202 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f1fa0-fe5e-48b7-8fed-c0b88864db88 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.093703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f1fa0-fe5e-48b7-8fed-c0b88864db88 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.093703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f1fa0-fe5e-48b7-8fed-c0b88864db88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.178145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d54f1fa0-fe5e-48b7-8fed-c0b88864db88 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:12.179820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d54f1fa0-fe5e-48b7-8fed-c0b88864db88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:12.297870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 676/1357] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:12.601557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852dd2-41ae-45e6-a8b0-2b647257585c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.602242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852dd2-41ae-45e6-a8b0-2b647257585c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.602737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852dd2-41ae-45e6-a8b0-2b647257585c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.604732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 682/1358] 10.222.0.86 () {54 vars in 962 bytes} [Mon Aug 29 18:54:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:12.612869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16baf8-e413-43f2-b43a-27ccab213c26 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.613503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16baf8-e413-43f2-b43a-27ccab213c26 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.613979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16baf8-e413-43f2-b43a-27ccab213c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.634942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:12.709801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5e16baf8-e413-43f2-b43a-27ccab213c26 None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:12.711225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e16baf8-e413-43f2-b43a-27ccab213c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:12.835463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 677/1359] 10.222.0.86 () {58 vars in 1059 bytes} [Mon Aug 29 18:54:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:12.843363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3852dd2-41ae-45e6-a8b0-2b647257585c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:12.844938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c3852dd2-41ae-45e6-a8b0-2b647257585c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:13.010911 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3852dd2-41ae-45e6-a8b0-2b647257585c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:13.091949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:13.134612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab707a-6b46-4a6d-80ec-4443054b6c10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:13.135443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab707a-6b46-4a6d-80ec-4443054b6c10 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:13.135824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab707a-6b46-4a6d-80ec-4443054b6c10 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:13.136197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab707a-6b46-4a6d-80ec-4443054b6c10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:13.137940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f3ab707a-6b46-4a6d-80ec-4443054b6c10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:13.230487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab707a-6b46-4a6d-80ec-4443054b6c10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:13.232107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab707a-6b46-4a6d-80ec-4443054b6c10 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:13.250800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 683/1360] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:15.390540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4aed8-35dd-4dd2-95b9-31c172017e95 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:15.391104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4aed8-35dd-4dd2-95b9-31c172017e95 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:15.391221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4aed8-35dd-4dd2-95b9-31c172017e95 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:15.392622 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 678/1361] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:15.397626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d220b-c400-4bd1-9377-7fcebab8f011 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:15.398069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d220b-c400-4bd1-9377-7fcebab8f011 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:15.398175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d220b-c400-4bd1-9377-7fcebab8f011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:15.467825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a03d220b-c400-4bd1-9377-7fcebab8f011 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:15.469038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a03d220b-c400-4bd1-9377-7fcebab8f011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:15.509986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:15.576353 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 684/1362] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:15.865569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4aed8-35dd-4dd2-95b9-31c172017e95 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:15.867735 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fca4aed8-35dd-4dd2-95b9-31c172017e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:15.938315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:16.031630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4aed8-35dd-4dd2-95b9-31c172017e95 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:16.152421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10372c6-07c2-44cb-80a6-7a9177386da1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:16.153041 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10372c6-07c2-44cb-80a6-7a9177386da1 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:16.153391 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10372c6-07c2-44cb-80a6-7a9177386da1 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:16.153722 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10372c6-07c2-44cb-80a6-7a9177386da1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:16.155434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e10372c6-07c2-44cb-80a6-7a9177386da1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:16.182360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:16.263288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10372c6-07c2-44cb-80a6-7a9177386da1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:16.265108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10372c6-07c2-44cb-80a6-7a9177386da1 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:16.284364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 679/1363] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:18.519569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdf560-4cb5-4f1e-ba46-b48a2935a19d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:18.520429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdf560-4cb5-4f1e-ba46-b48a2935a19d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:18.520709 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdf560-4cb5-4f1e-ba46-b48a2935a19d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:18.522061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 685/1364] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:18.528525 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b814d-54eb-425d-a93a-87145f56deeb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:18.528991 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b814d-54eb-425d-a93a-87145f56deeb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:18.529186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b814d-54eb-425d-a93a-87145f56deeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:18.613301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-562b814d-54eb-425d-a93a-87145f56deeb None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:18.614679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-562b814d-54eb-425d-a93a-87145f56deeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:18.721520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:18.730981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 680/1365] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:19.018512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306844ba-7683-46df-a852-1f4e2b833c85 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:19.018818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306844ba-7683-46df-a852-1f4e2b833c85 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:19.019309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306844ba-7683-46df-a852-1f4e2b833c85 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:19.021328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 686/1366] 10.222.0.86 () {54 vars in 962 bytes} [Mon Aug 29 18:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:19.026555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6de3-fa7f-4878-a1ca-a83947b28bf5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:19.027191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6de3-fa7f-4878-a1ca-a83947b28bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:19.027569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6de3-fa7f-4878-a1ca-a83947b28bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:19.124751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-26ad6de3-fa7f-4878-a1ca-a83947b28bf5 None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:19.126134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-26ad6de3-fa7f-4878-a1ca-a83947b28bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:19.248077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 681/1367] 10.222.0.86 () {58 vars in 1059 bytes} [Mon Aug 29 18:54:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:19.256220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306844ba-7683-46df-a852-1f4e2b833c85 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:19.257920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-306844ba-7683-46df-a852-1f4e2b833c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:19.425811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306844ba-7683-46df-a852-1f4e2b833c85 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:19.447452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:19.549770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edf0c5-c305-44fd-98b2-2c4ae676eb35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:19.550397 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edf0c5-c305-44fd-98b2-2c4ae676eb35 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:19.550787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edf0c5-c305-44fd-98b2-2c4ae676eb35 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:19.551188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edf0c5-c305-44fd-98b2-2c4ae676eb35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:19.552882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c9edf0c5-c305-44fd-98b2-2c4ae676eb35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:19.651377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edf0c5-c305-44fd-98b2-2c4ae676eb35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:19.653127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edf0c5-c305-44fd-98b2-2c4ae676eb35 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:19.672653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 687/1368] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:20.341470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d54128-34b4-4089-a42a-345d81981c7d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:20.342004 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d54128-34b4-4089-a42a-345d81981c7d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:20.342004 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d54128-34b4-4089-a42a-345d81981c7d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:20.343546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 682/1369] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:20.348688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b27d7-2e13-4bee-8b4f-39c22cb47cd0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:20.348979 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b27d7-2e13-4bee-8b4f-39c22cb47cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:20.349257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b27d7-2e13-4bee-8b4f-39c22cb47cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:20.384739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:20.422580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-089b27d7-2e13-4bee-8b4f-39c22cb47cd0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:20.423806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-089b27d7-2e13-4bee-8b4f-39c22cb47cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:20.524903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 688/1370] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:54:21.112635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c04ecd-c9fd-446d-96dc-89a22e2a1ee1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.112635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c04ecd-c9fd-446d-96dc-89a22e2a1ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.112635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c04ecd-c9fd-446d-96dc-89a22e2a1ee1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.114478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 683/1371] 10.222.0.86 () {54 vars in 964 bytes} [Mon Aug 29 18:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:21.119231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa5bce-5162-4620-9722-00eb6bc1d7c1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.119763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa5bce-5162-4620-9722-00eb6bc1d7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.119763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa5bce-5162-4620-9722-00eb6bc1d7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.201991 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-81aa5bce-5162-4620-9722-00eb6bc1d7c1 None None] MFA Rules not processed for user `a3030e2c49614aee892c7693da6b73bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:21.203232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-81aa5bce-5162-4620-9722-00eb6bc1d7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:21.254118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:21.287580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 689/1372] 10.222.0.86 () {58 vars in 1061 bytes} [Mon Aug 29 18:54:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:21.295492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c04ecd-c9fd-446d-96dc-89a22e2a1ee1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:21.297399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c6c04ecd-c9fd-446d-96dc-89a22e2a1ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:21.357642 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:21.416370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c04ecd-c9fd-446d-96dc-89a22e2a1ee1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:21.499216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ae970-6940-4b29-b283-06742e5e76d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:21.499887 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ae970-6940-4b29-b283-06742e5e76d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.500175 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ae970-6940-4b29-b283-06742e5e76d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.500443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ae970-6940-4b29-b283-06742e5e76d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.502104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-be3ae970-6940-4b29-b283-06742e5e76d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:21.641705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:21.658141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ae970-6940-4b29-b283-06742e5e76d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:21.659640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ae970-6940-4b29-b283-06742e5e76d1 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:21.676649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5e997-19fe-4321-a9eb-f43042d19aa5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.676862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5e997-19fe-4321-a9eb-f43042d19aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.677145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5e997-19fe-4321-a9eb-f43042d19aa5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.678501 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 690/1373] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:54:21.685695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab8a9c-8730-40e9-972e-0274afedc0c3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.685995 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab8a9c-8730-40e9-972e-0274afedc0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.686438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab8a9c-8730-40e9-972e-0274afedc0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.693093 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 684/1374] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 18:54:21.764589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a3ab8a9c-8730-40e9-972e-0274afedc0c3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:21.765694 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a3ab8a9c-8730-40e9-972e-0274afedc0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:21.867854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 691/1375] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:22.154973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ae970-6940-4b29-b283-06742e5e76d1 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:22.156301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-be3ae970-6940-4b29-b283-06742e5e76d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:22.254172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ae970-6940-4b29-b283-06742e5e76d1 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:22.355396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:22.368257 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f5205-ed2f-4716-b6d4-dcb852a8609d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:22.368965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f5205-ed2f-4716-b6d4-dcb852a8609d service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.369131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f5205-ed2f-4716-b6d4-dcb852a8609d service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.369405 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f5205-ed2f-4716-b6d4-dcb852a8609d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.371022 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a58f5205-ed2f-4716-b6d4-dcb852a8609d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:22.464986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f5205-ed2f-4716-b6d4-dcb852a8609d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:22.466719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f5205-ed2f-4716-b6d4-dcb852a8609d service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:22.484401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 685/1376] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:24.452119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b50e-8d2a-4e81-8fc9-d7dec20f27b5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:24.452658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b50e-8d2a-4e81-8fc9-d7dec20f27b5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:24.452658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b50e-8d2a-4e81-8fc9-d7dec20f27b5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:24.454092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 692/1377] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:24.459262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f7674-f488-46db-81c1-d885a34a62c0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:24.459669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f7674-f488-46db-81c1-d885a34a62c0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:24.459825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f7674-f488-46db-81c1-d885a34a62c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:24.526396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9b5f7674-f488-46db-81c1-d885a34a62c0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:24.527643 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9b5f7674-f488-46db-81c1-d885a34a62c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:24.592393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:24.631397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 686/1378] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:24.947298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b50e-8d2a-4e81-8fc9-d7dec20f27b5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:24.948805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-16e8b50e-8d2a-4e81-8fc9-d7dec20f27b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:25.001394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:25.103362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b50e-8d2a-4e81-8fc9-d7dec20f27b5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:25.243900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afb22f-25c9-45c4-bee1-1619597c6adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:25.244521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afb22f-25c9-45c4-bee1-1619597c6adc service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:25.244804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afb22f-25c9-45c4-bee1-1619597c6adc service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:25.245077 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afb22f-25c9-45c4-bee1-1619597c6adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:25.246669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a6afb22f-25c9-45c4-bee1-1619597c6adc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:25.267133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:25.368621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afb22f-25c9-45c4-bee1-1619597c6adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:25.370321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afb22f-25c9-45c4-bee1-1619597c6adc service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:25.395211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 693/1379] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:27.377788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae1dec-4c2b-4b46-bfa8-a51e8e0d87bc None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:27.378342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae1dec-4c2b-4b46-bfa8-a51e8e0d87bc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:27.378342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae1dec-4c2b-4b46-bfa8-a51e8e0d87bc None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:27.380017 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 687/1380] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:27.386056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734e401-a7dd-4133-90ef-82a897f319b0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:27.386655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734e401-a7dd-4133-90ef-82a897f319b0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:27.386831 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734e401-a7dd-4133-90ef-82a897f319b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:27.457650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e734e401-a7dd-4133-90ef-82a897f319b0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:27.459051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e734e401-a7dd-4133-90ef-82a897f319b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:27.549276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:27.563343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 694/1381] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:27.855948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae1dec-4c2b-4b46-bfa8-a51e8e0d87bc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:27.857667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b9ae1dec-4c2b-4b46-bfa8-a51e8e0d87bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:28.021083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae1dec-4c2b-4b46-bfa8-a51e8e0d87bc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:28.079789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:28.166033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b23382-8e02-4128-971c-1ffa4a01d4e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:28.166677 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b23382-8e02-4128-971c-1ffa4a01d4e7 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:28.166956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b23382-8e02-4128-971c-1ffa4a01d4e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:28.167255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b23382-8e02-4128-971c-1ffa4a01d4e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:28.169072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-34b23382-8e02-4128-971c-1ffa4a01d4e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:28.268451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b23382-8e02-4128-971c-1ffa4a01d4e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:28.270044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b23382-8e02-4128-971c-1ffa4a01d4e7 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:28.299206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 688/1382] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:30.268835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45a53c-dc6c-40f2-9bab-bc0450eff92a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:30.269494 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45a53c-dc6c-40f2-9bab-bc0450eff92a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:30.269494 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45a53c-dc6c-40f2-9bab-bc0450eff92a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:30.270793 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 695/1383] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:30.277166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0fde8-6ece-4024-b7b8-a7764d92f615 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:30.277578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0fde8-6ece-4024-b7b8-a7764d92f615 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:30.277864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0fde8-6ece-4024-b7b8-a7764d92f615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:30.351056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ebc0fde8-6ece-4024-b7b8-a7764d92f615 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:30.352874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ebc0fde8-6ece-4024-b7b8-a7764d92f615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:30.355080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:30.457131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 689/1384] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:54:30.762091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45a53c-dc6c-40f2-9bab-bc0450eff92a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:30.763659 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-de45a53c-dc6c-40f2-9bab-bc0450eff92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:30.937406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45a53c-dc6c-40f2-9bab-bc0450eff92a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:31.025425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:31.077593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69187932-888f-403c-a280-407b9d681581 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:31.078190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187932-888f-403c-a280-407b9d681581 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:31.078470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187932-888f-403c-a280-407b9d681581 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:31.079044 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187932-888f-403c-a280-407b9d681581 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:31.080369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-69187932-888f-403c-a280-407b9d681581 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:31.187204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69187932-888f-403c-a280-407b9d681581 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:31.190968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69187932-888f-403c-a280-407b9d681581 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:31.221226 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 696/1385] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:33.207858 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590bfe7-ba08-4b05-bdee-2e3cc063da12 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.208507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590bfe7-ba08-4b05-bdee-2e3cc063da12 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.208507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590bfe7-ba08-4b05-bdee-2e3cc063da12 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.209770 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 690/1386] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:33.215727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8a526-071b-4116-a96a-7db530e0cf88 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.216267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8a526-071b-4116-a96a-7db530e0cf88 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.216267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8a526-071b-4116-a96a-7db530e0cf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.284734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7cb8a526-071b-4116-a96a-7db530e0cf88 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:33.285966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7cb8a526-071b-4116-a96a-7db530e0cf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:33.314365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.408323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 697/1387] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:54:33.708567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590bfe7-ba08-4b05-bdee-2e3cc063da12 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:33.710224 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0590bfe7-ba08-4b05-bdee-2e3cc063da12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:33.842319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.859025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590bfe7-ba08-4b05-bdee-2e3cc063da12 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.976683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccdd890-f129-4a3a-883c-a71457171f5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.977287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdd890-f129-4a3a-883c-a71457171f5c service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.977684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdd890-f129-4a3a-883c-a71457171f5c service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.977969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdd890-f129-4a3a-883c-a71457171f5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.979848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1ccdd890-f129-4a3a-883c-a71457171f5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:34.081336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccdd890-f129-4a3a-883c-a71457171f5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:34.083396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccdd890-f129-4a3a-883c-a71457171f5c service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:34.089177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:34.103902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 691/1388] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:36.324291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355121ef-5cb7-4c28-b4af-04a2d19d9d5d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:36.325145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355121ef-5cb7-4c28-b4af-04a2d19d9d5d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:36.325145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355121ef-5cb7-4c28-b4af-04a2d19d9d5d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:36.326443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 698/1389] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:36.331665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c0aca-9595-4d26-9209-08ab65bed729 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:36.332111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c0aca-9595-4d26-9209-08ab65bed729 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:36.332292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c0aca-9595-4d26-9209-08ab65bed729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:36.400930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-402c0aca-9595-4d26-9209-08ab65bed729 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:36.402147 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-402c0aca-9595-4d26-9209-08ab65bed729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:36.501924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 692/1390] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:36.791555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355121ef-5cb7-4c28-b4af-04a2d19d9d5d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:36.793372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-355121ef-5cb7-4c28-b4af-04a2d19d9d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:36.945338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355121ef-5cb7-4c28-b4af-04a2d19d9d5d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:36.954883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:37.060265 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579f7dc-6b5a-494c-b44d-09a0be2637d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:37.060872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f7dc-6b5a-494c-b44d-09a0be2637d3 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:37.061147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f7dc-6b5a-494c-b44d-09a0be2637d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:37.061416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f7dc-6b5a-494c-b44d-09a0be2637d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:37.062984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6579f7dc-6b5a-494c-b44d-09a0be2637d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:37.153384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579f7dc-6b5a-494c-b44d-09a0be2637d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:37.155092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579f7dc-6b5a-494c-b44d-09a0be2637d3 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:37.173152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 699/1391] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:38.111239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943af14-bdfd-4b41-813e-306558f79b20 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.111822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943af14-bdfd-4b41-813e-306558f79b20 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.111822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943af14-bdfd-4b41-813e-306558f79b20 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.113102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 693/1392] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:38.120037 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4c74-434e-41ec-890f-a52fe1c33bf7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.120450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4c74-434e-41ec-890f-a52fe1c33bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.120750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4c74-434e-41ec-890f-a52fe1c33bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.136335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:38.188621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-70cf4c74-434e-41ec-890f-a52fe1c33bf7 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:38.189823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-70cf4c74-434e-41ec-890f-a52fe1c33bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:38.287873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 700/1393] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:38.592600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adcbfd-10ac-4cc9-a93c-0b867a4a3dc1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.593018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adcbfd-10ac-4cc9-a93c-0b867a4a3dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.593382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adcbfd-10ac-4cc9-a93c-0b867a4a3dc1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.595172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 694/1394] 10.222.0.86 () {54 vars in 967 bytes} [Mon Aug 29 18:54:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:38.600678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cc92d-09a3-4943-bb27-bceeb3d18db2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.601101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cc92d-09a3-4943-bb27-bceeb3d18db2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.602029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cc92d-09a3-4943-bb27-bceeb3d18db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.691771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-916cc92d-09a3-4943-bb27-bceeb3d18db2 None None] MFA Rules not processed for user `9af69888b04c45b09e34d9ff7a0b564f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:38.693040 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-916cc92d-09a3-4943-bb27-bceeb3d18db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:38.727844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:38.774391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 701/1395] 10.222.0.86 () {58 vars in 1064 bytes} [Mon Aug 29 18:54:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:38.782076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adcbfd-10ac-4cc9-a93c-0b867a4a3dc1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:38.783638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-19adcbfd-10ac-4cc9-a93c-0b867a4a3dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:38.856673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:38.888744 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adcbfd-10ac-4cc9-a93c-0b867a4a3dc1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:38.965469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fdaaf-8370-4965-81bd-01b5a400fc03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af69888b04c45b09e34d9ff7a0b564f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:38.966260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fdaaf-8370-4965-81bd-01b5a400fc03 service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.966394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fdaaf-8370-4965-81bd-01b5a400fc03 service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.966637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fdaaf-8370-4965-81bd-01b5a400fc03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.968301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-029fdaaf-8370-4965-81bd-01b5a400fc03 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:39.079190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fdaaf-8370-4965-81bd-01b5a400fc03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:39.080871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fdaaf-8370-4965-81bd-01b5a400fc03 service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:39.097447 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:39.101788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 695/1396] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:39.127384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c0a58-7309-4c4d-8e39-57c43089f663 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.127893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c0a58-7309-4c4d-8e39-57c43089f663 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.127893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c0a58-7309-4c4d-8e39-57c43089f663 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.129238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 702/1397] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:39.134472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8a1e7-7848-43a7-85fc-bc6d00d7d443 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.134906 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8a1e7-7848-43a7-85fc-bc6d00d7d443 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.135052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8a1e7-7848-43a7-85fc-bc6d00d7d443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.200785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1fb8a1e7-7848-43a7-85fc-bc6d00d7d443 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:39.201994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1fb8a1e7-7848-43a7-85fc-bc6d00d7d443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:39.298811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 696/1398] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:54:39.611657 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c0a58-7309-4c4d-8e39-57c43089f663 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.613279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-244c0a58-7309-4c4d-8e39-57c43089f663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:39.789086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c0a58-7309-4c4d-8e39-57c43089f663 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:39.830730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:39.871670 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642269f-8020-437a-90b8-80fa8aa491c1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.871956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642269f-8020-437a-90b8-80fa8aa491c1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.872233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642269f-8020-437a-90b8-80fa8aa491c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.913340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d766298-6069-4910-90c0-e80fd761ff22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.914595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d766298-6069-4910-90c0-e80fd761ff22 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.914923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d766298-6069-4910-90c0-e80fd761ff22 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.915362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d766298-6069-4910-90c0-e80fd761ff22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.917015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8d766298-6069-4910-90c0-e80fd761ff22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:39.943553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8642269f-8020-437a-90b8-80fa8aa491c1 None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:39.944763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8642269f-8020-437a-90b8-80fa8aa491c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:39.976266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:40.009615 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d766298-6069-4910-90c0-e80fd761ff22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:40.011237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d766298-6069-4910-90c0-e80fd761ff22 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:40.030340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 703/1399] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:40.033893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 697/1400] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 18:54:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:54:40.080958 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ea1a4-036e-4da5-a037-7aac54ff5eb8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.081457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ea1a4-036e-4da5-a037-7aac54ff5eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.082035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ea1a4-036e-4da5-a037-7aac54ff5eb8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.083606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 704/1401] 10.222.0.86 () {54 vars in 943 bytes} [Mon Aug 29 18:54:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:40.094083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204cdbd-5d9d-442a-9584-1af1ae00dfc3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.094375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204cdbd-5d9d-442a-9584-1af1ae00dfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.094655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204cdbd-5d9d-442a-9584-1af1ae00dfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.181594 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6204cdbd-5d9d-442a-9584-1af1ae00dfc3 None None] MFA Rules not processed for user `dd1b7173843b4da48e69da8b36d59226`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:40.182872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6204cdbd-5d9d-442a-9584-1af1ae00dfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:40.295831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 698/1402] 10.222.0.86 () {58 vars in 1040 bytes} [Mon Aug 29 18:54:40 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:54:40.305047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ea1a4-036e-4da5-a037-7aac54ff5eb8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:40.306569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-734ea1a4-036e-4da5-a037-7aac54ff5eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:40.356837 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:40.444714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ea1a4-036e-4da5-a037-7aac54ff5eb8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:40.548568 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d09e4f-431f-4b5a-bc6c-bd8fa75ac543 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:40.549093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d09e4f-431f-4b5a-bc6c-bd8fa75ac543 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.549416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d09e4f-431f-4b5a-bc6c-bd8fa75ac543 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.549690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d09e4f-431f-4b5a-bc6c-bd8fa75ac543 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.551331 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-61d09e4f-431f-4b5a-bc6c-bd8fa75ac543 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:40.582229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:40.638074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d09e4f-431f-4b5a-bc6c-bd8fa75ac543 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:40.639986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d09e4f-431f-4b5a-bc6c-bd8fa75ac543 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:40.648038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 705/1403] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 18:54:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 18:54:42.108008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826efafe-32bd-44a9-8fc9-fd31d99b5e2c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.108541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826efafe-32bd-44a9-8fc9-fd31d99b5e2c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.108541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826efafe-32bd-44a9-8fc9-fd31d99b5e2c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.113790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 699/1404] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:42.115112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce278-8ca0-470f-9eb6-2e7bb88d5206 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.115534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce278-8ca0-470f-9eb6-2e7bb88d5206 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.115725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce278-8ca0-470f-9eb6-2e7bb88d5206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.193218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8dbce278-8ca0-470f-9eb6-2e7bb88d5206 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:42.193584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8dbce278-8ca0-470f-9eb6-2e7bb88d5206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:42.369248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 706/1405] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:54:42.666507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826efafe-32bd-44a9-8fc9-fd31d99b5e2c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:42.669913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-826efafe-32bd-44a9-8fc9-fd31d99b5e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:42.674008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:42.843168 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826efafe-32bd-44a9-8fc9-fd31d99b5e2c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:42.925659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:42.975984 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b2acb-23ae-4429-a8f3-0fe390286041 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:42.976760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b2acb-23ae-4429-a8f3-0fe390286041 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.977121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b2acb-23ae-4429-a8f3-0fe390286041 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.977524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b2acb-23ae-4429-a8f3-0fe390286041 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.982288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2b0b2acb-23ae-4429-a8f3-0fe390286041 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:43.085112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b2acb-23ae-4429-a8f3-0fe390286041 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:43.087026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b2acb-23ae-4429-a8f3-0fe390286041 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:43.109705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 700/1406] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:45.091625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c9d63-00d5-4312-8b40-3f714b8f1ff4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.092205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c9d63-00d5-4312-8b40-3f714b8f1ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.092205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c9d63-00d5-4312-8b40-3f714b8f1ff4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.093494 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 707/1407] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:45.098698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730e0a5-57eb-426c-8cd2-51e278d2175c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.099131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730e0a5-57eb-426c-8cd2-51e278d2175c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.099380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730e0a5-57eb-426c-8cd2-51e278d2175c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.170600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0730e0a5-57eb-426c-8cd2-51e278d2175c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:45.171839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0730e0a5-57eb-426c-8cd2-51e278d2175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:45.214674 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:45.280427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 701/1408] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:54:45.578355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c9d63-00d5-4312-8b40-3f714b8f1ff4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:45.580187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ec3c9d63-00d5-4312-8b40-3f714b8f1ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:45.608337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:45.746886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c9d63-00d5-4312-8b40-3f714b8f1ff4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:45.860874 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:45.882113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763084a-381f-4716-b5ec-61e08cbd71eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:45.883427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763084a-381f-4716-b5ec-61e08cbd71eb service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.883804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763084a-381f-4716-b5ec-61e08cbd71eb service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.884184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763084a-381f-4716-b5ec-61e08cbd71eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.885949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a763084a-381f-4716-b5ec-61e08cbd71eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:45.991014 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763084a-381f-4716-b5ec-61e08cbd71eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:45.993747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763084a-381f-4716-b5ec-61e08cbd71eb service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:46.015866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 708/1409] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:54:55.278366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=26 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:56.013225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=52 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:03.951995 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aeff0-50d9-41d8-85e1-e17e6acbe2a5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:03.952588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aeff0-50d9-41d8-85e1-e17e6acbe2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:03.952588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aeff0-50d9-41d8-85e1-e17e6acbe2a5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:03.953794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 702/1410] 10.222.0.86 () {54 vars in 918 bytes} [Mon Aug 29 18:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:03.958879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1a091-7614-4ed9-a541-9fd02dfe5132 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:03.959342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1a091-7614-4ed9-a541-9fd02dfe5132 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:03.959472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1a091-7614-4ed9-a541-9fd02dfe5132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:04.034791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-92c1a091-7614-4ed9-a541-9fd02dfe5132 None None] MFA Rules not processed for user `27f771f20b734b679d5553d29a8a3884`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:04.036030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-92c1a091-7614-4ed9-a541-9fd02dfe5132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:04.142363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 709/1411] 10.222.0.86 () {58 vars in 1015 bytes} [Mon Aug 29 18:55:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:55:04.157292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5f8fd-1427-4c67-93ae-cb842c115d7f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:04.157578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5f8fd-1427-4c67-93ae-cb842c115d7f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:04.157855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5f8fd-1427-4c67-93ae-cb842c115d7f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:04.159187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 703/1412] 10.222.0.86 () {54 vars in 918 bytes} [Mon Aug 29 18:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:04.164456 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfcf75-ad41-4bde-bd6d-84ce645f48c2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:04.164737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfcf75-ad41-4bde-bd6d-84ce645f48c2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:04.165012 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfcf75-ad41-4bde-bd6d-84ce645f48c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:04.239528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:04.242897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2ebfcf75-ad41-4bde-bd6d-84ce645f48c2 None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:04.244696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2ebfcf75-ad41-4bde-bd6d-84ce645f48c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:04.361029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 710/1413] 10.222.0.86 () {58 vars in 1015 bytes} [Mon Aug 29 18:55:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:06.754035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eba75c-b71e-4a45-a54b-f47d408e359e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:06.754562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eba75c-b71e-4a45-a54b-f47d408e359e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:06.754562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eba75c-b71e-4a45-a54b-f47d408e359e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:06.755934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:06.761842 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34ae1a-ed56-46b8-bff9-6e9e573033d9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:06.762267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34ae1a-ed56-46b8-bff9-6e9e573033d9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:06.762371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34ae1a-ed56-46b8-bff9-6e9e573033d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:06.827169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ad34ae1a-ed56-46b8-bff9-6e9e573033d9 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:06.828352 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ad34ae1a-ed56-46b8-bff9-6e9e573033d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:06.881351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:06.924084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:07.215557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eba75c-b71e-4a45-a54b-f47d408e359e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:07.217244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d6eba75c-b71e-4a45-a54b-f47d408e359e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:07.362821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eba75c-b71e-4a45-a54b-f47d408e359e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:07.434599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:07.474492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd5190-d659-4479-95da-1a86941e09cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:07.475046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd5190-d659-4479-95da-1a86941e09cf service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:07.475328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd5190-d659-4479-95da-1a86941e09cf service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:07.476144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd5190-d659-4479-95da-1a86941e09cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:07.477757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-69dd5190-d659-4479-95da-1a86941e09cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:07.566109 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd5190-d659-4479-95da-1a86941e09cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:07.567819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd5190-d659-4479-95da-1a86941e09cf service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:07.585816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 705/1416] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:08.474623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728143a1-42f3-4e87-8121-47068ba1dcaa None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:08.475247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728143a1-42f3-4e87-8121-47068ba1dcaa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:08.475247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728143a1-42f3-4e87-8121-47068ba1dcaa None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:08.476597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:08.481700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8635c9d-b30b-481e-acdd-6b307f52ff44 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:08.482199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8635c9d-b30b-481e-acdd-6b307f52ff44 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:08.482302 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8635c9d-b30b-481e-acdd-6b307f52ff44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:08.557616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c8635c9d-b30b-481e-acdd-6b307f52ff44 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:08.558823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c8635c9d-b30b-481e-acdd-6b307f52ff44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:08.587697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:08.656787 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:08.807457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728143a1-42f3-4e87-8121-47068ba1dcaa None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:08.808994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-728143a1-42f3-4e87-8121-47068ba1dcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:08.959755 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728143a1-42f3-4e87-8121-47068ba1dcaa None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:09.004306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:09.077987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846adef7-1c92-420a-9c96-2dfde79139af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:09.078505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846adef7-1c92-420a-9c96-2dfde79139af service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.078780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846adef7-1c92-420a-9c96-2dfde79139af service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.079071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846adef7-1c92-420a-9c96-2dfde79139af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.080648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-846adef7-1c92-420a-9c96-2dfde79139af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:09.170305 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846adef7-1c92-420a-9c96-2dfde79139af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:09.171907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846adef7-1c92-420a-9c96-2dfde79139af service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:09.190754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 713/1419] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:10.075961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b5527-5e26-4f15-aea7-2d032e279d74 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:10.076497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b5527-5e26-4f15-aea7-2d032e279d74 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:10.076497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b5527-5e26-4f15-aea7-2d032e279d74 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:10.077829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:10.083390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92461d9-39a9-4a86-81ce-873d35eff4c9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:10.083839 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92461d9-39a9-4a86-81ce-873d35eff4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:10.084012 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92461d9-39a9-4a86-81ce-873d35eff4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:10.138492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:10.159817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d92461d9-39a9-4a86-81ce-873d35eff4c9 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:10.161006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d92461d9-39a9-4a86-81ce-873d35eff4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:10.256998 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:55:10.410628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b5527-5e26-4f15-aea7-2d032e279d74 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:10.412495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-212b5527-5e26-4f15-aea7-2d032e279d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:10.559671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b5527-5e26-4f15-aea7-2d032e279d74 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:10.572130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:10.673227 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d453ec-f327-43bd-9e44-d03e630de3fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:10.673747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d453ec-f327-43bd-9e44-d03e630de3fd service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:10.674020 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d453ec-f327-43bd-9e44-d03e630de3fd service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:10.674287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d453ec-f327-43bd-9e44-d03e630de3fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:10.675900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-85d453ec-f327-43bd-9e44-d03e630de3fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:10.764895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d453ec-f327-43bd-9e44-d03e630de3fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:10.766466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d453ec-f327-43bd-9e44-d03e630de3fd service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:10.784662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 708/1422] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:11.668938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb16a-42fe-4afb-9c5c-025fcb9ffb91 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:11.669474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb16a-42fe-4afb-9c5c-025fcb9ffb91 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:11.669474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb16a-42fe-4afb-9c5c-025fcb9ffb91 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:11.670808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:11.675917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f71d3e-15fa-44a6-b7b8-3c92b7505c1f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:11.676384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f71d3e-15fa-44a6-b7b8-3c92b7505c1f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:11.676648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f71d3e-15fa-44a6-b7b8-3c92b7505c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:11.696331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:11.753184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d9f71d3e-15fa-44a6-b7b8-3c92b7505c1f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:11.754399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d9f71d3e-15fa-44a6-b7b8-3c92b7505c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:11.849034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:12.000239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddb16a-42fe-4afb-9c5c-025fcb9ffb91 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:12.001732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f6ddb16a-42fe-4afb-9c5c-025fcb9ffb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:12.109340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:12.145047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddb16a-42fe-4afb-9c5c-025fcb9ffb91 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:12.256140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f02826-4bb6-437d-a214-2dca771269f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:12.256744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f02826-4bb6-437d-a214-2dca771269f4 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:12.257025 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f02826-4bb6-437d-a214-2dca771269f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:12.257294 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f02826-4bb6-437d-a214-2dca771269f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:12.258877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-38f02826-4bb6-437d-a214-2dca771269f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:12.333601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:12.347977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f02826-4bb6-437d-a214-2dca771269f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:12.349567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f02826-4bb6-437d-a214-2dca771269f4 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:12.367889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 716/1425] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:13.254321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87d9ab-f9cc-4c59-9ec1-8a8cf458a55a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.254888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87d9ab-f9cc-4c59-9ec1-8a8cf458a55a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.254888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87d9ab-f9cc-4c59-9ec1-8a8cf458a55a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.256159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:13.261639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3caa6-7e32-4a2d-bad1-1488d31ae9f0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.262081 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3caa6-7e32-4a2d-bad1-1488d31ae9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.262184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3caa6-7e32-4a2d-bad1-1488d31ae9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.338210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-eae3caa6-7e32-4a2d-bad1-1488d31ae9f0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:13.339439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eae3caa6-7e32-4a2d-bad1-1488d31ae9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:13.436924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:13.592190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87d9ab-f9cc-4c59-9ec1-8a8cf458a55a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:13.593740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bf87d9ab-f9cc-4c59-9ec1-8a8cf458a55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:13.678303 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:13.742226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87d9ab-f9cc-4c59-9ec1-8a8cf458a55a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:13.853867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea61984-02da-44e3-a3d7-f554efb6cfcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:13.854399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea61984-02da-44e3-a3d7-f554efb6cfcf service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.854681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea61984-02da-44e3-a3d7-f554efb6cfcf service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.854952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea61984-02da-44e3-a3d7-f554efb6cfcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.856606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8ea61984-02da-44e3-a3d7-f554efb6cfcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:13.903366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:13.944838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea61984-02da-44e3-a3d7-f554efb6cfcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:13.946486 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea61984-02da-44e3-a3d7-f554efb6cfcf service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:13.965124 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 711/1428] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:14.771157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c18fc-d310-4a0c-a73d-a2888fc22dfd None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:14.771658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c18fc-d310-4a0c-a73d-a2888fc22dfd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:14.771771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c18fc-d310-4a0c-a73d-a2888fc22dfd None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:14.773104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:14.778182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ca74f-e6ee-426d-b1b0-cef13ef1eeef None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:14.778613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ca74f-e6ee-426d-b1b0-cef13ef1eeef None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:14.778723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ca74f-e6ee-426d-b1b0-cef13ef1eeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:14.854744 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a92ca74f-e6ee-426d-b1b0-cef13ef1eeef None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:14.855959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a92ca74f-e6ee-426d-b1b0-cef13ef1eeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:14.951750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:15.106648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c18fc-d310-4a0c-a73d-a2888fc22dfd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:15.108233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b86c18fc-d310-4a0c-a73d-a2888fc22dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:15.156748 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:15.252808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c18fc-d310-4a0c-a73d-a2888fc22dfd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:15.366259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52dbd4-d906-431b-bf4f-7c5b0c39bbf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:15.366867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52dbd4-d906-431b-bf4f-7c5b0c39bbf8 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:15.367167 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52dbd4-d906-431b-bf4f-7c5b0c39bbf8 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:15.367433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52dbd4-d906-431b-bf4f-7c5b0c39bbf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:15.369010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ce52dbd4-d906-431b-bf4f-7c5b0c39bbf8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:15.375828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:15.458582 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52dbd4-d906-431b-bf4f-7c5b0c39bbf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:15.461036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52dbd4-d906-431b-bf4f-7c5b0c39bbf8 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:15.479636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 719/1431] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:16.300352 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e36b15-3f58-412d-9fc2-77e951037ae5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:16.300925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e36b15-3f58-412d-9fc2-77e951037ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:16.300925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e36b15-3f58-412d-9fc2-77e951037ae5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:16.302251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:16.307647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b52a9a-2750-4f4d-a3fa-f6432b3fa572 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:16.307647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b52a9a-2750-4f4d-a3fa-f6432b3fa572 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:16.308390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b52a9a-2750-4f4d-a3fa-f6432b3fa572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:16.384419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d8b52a9a-2750-4f4d-a3fa-f6432b3fa572 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:16.385658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d8b52a9a-2750-4f4d-a3fa-f6432b3fa572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:16.454258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:16.484162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:16.636433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e36b15-3f58-412d-9fc2-77e951037ae5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:16.638118 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d9e36b15-3f58-412d-9fc2-77e951037ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:16.656101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:16.786681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e36b15-3f58-412d-9fc2-77e951037ae5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:16.872422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:16.899303 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403a66a-7b83-4b7b-87b9-16f51f40b075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:16.899850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a66a-7b83-4b7b-87b9-16f51f40b075 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:16.900129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a66a-7b83-4b7b-87b9-16f51f40b075 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:16.900401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a66a-7b83-4b7b-87b9-16f51f40b075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:16.902012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f403a66a-7b83-4b7b-87b9-16f51f40b075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:16.989900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403a66a-7b83-4b7b-87b9-16f51f40b075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:16.991563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403a66a-7b83-4b7b-87b9-16f51f40b075 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:17.009192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 714/1434] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:17.810511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccc1a5-0587-463c-93d5-3b6f609f194b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:17.811128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccc1a5-0587-463c-93d5-3b6f609f194b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:17.811128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccc1a5-0587-463c-93d5-3b6f609f194b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:17.812476 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:17.817597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361b3c5-5944-4d45-95a2-94497111c2ff None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:17.818026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361b3c5-5944-4d45-95a2-94497111c2ff None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:17.818135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361b3c5-5944-4d45-95a2-94497111c2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:17.894290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3361b3c5-5944-4d45-95a2-94497111c2ff None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:17.895460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3361b3c5-5944-4d45-95a2-94497111c2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:17.939619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:17.995317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:55:18.148407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccc1a5-0587-463c-93d5-3b6f609f194b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:18.149518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-99ccc1a5-0587-463c-93d5-3b6f609f194b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:18.297601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccc1a5-0587-463c-93d5-3b6f609f194b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:18.353438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:18.412670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540d612-5eef-4798-ad10-449a37401b32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:18.413267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d612-5eef-4798-ad10-449a37401b32 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:18.413512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d612-5eef-4798-ad10-449a37401b32 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:18.413781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d612-5eef-4798-ad10-449a37401b32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:18.415384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a540d612-5eef-4798-ad10-449a37401b32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:18.504639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540d612-5eef-4798-ad10-449a37401b32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:18.506219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540d612-5eef-4798-ad10-449a37401b32 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:18.524587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 722/1437] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:19.330508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf0779-e282-4d74-889b-740d4b738888 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.331102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf0779-e282-4d74-889b-740d4b738888 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.331102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf0779-e282-4d74-889b-740d4b738888 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.332433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:19.337795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78e5ed-459a-4b16-99ef-50ecc807e57e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.338198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78e5ed-459a-4b16-99ef-50ecc807e57e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.338338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78e5ed-459a-4b16-99ef-50ecc807e57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.413554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1f78e5ed-459a-4b16-99ef-50ecc807e57e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:19.417310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1f78e5ed-459a-4b16-99ef-50ecc807e57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:19.419427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:19.522932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:19.673713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf0779-e282-4d74-889b-740d4b738888 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:19.675317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d8bf0779-e282-4d74-889b-740d4b738888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:19.822960 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf0779-e282-4d74-889b-740d4b738888 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:19.850851 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:19.937246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747df80-7e6c-4e99-89a2-77e0cb9f186a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:19.937814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747df80-7e6c-4e99-89a2-77e0cb9f186a service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.938094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747df80-7e6c-4e99-89a2-77e0cb9f186a service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.938365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747df80-7e6c-4e99-89a2-77e0cb9f186a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.940016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c747df80-7e6c-4e99-89a2-77e0cb9f186a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:20.028410 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747df80-7e6c-4e99-89a2-77e0cb9f186a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:20.030009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747df80-7e6c-4e99-89a2-77e0cb9f186a service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:20.048114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 717/1440] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:21.872090 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8d000-097a-4c53-a4a1-394760dd2793 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:21.872673 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8d000-097a-4c53-a4a1-394760dd2793 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:21.872673 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8d000-097a-4c53-a4a1-394760dd2793 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:21.873985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:21.879443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42faee-9209-4acb-ad60-f68c24323728 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:21.879869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42faee-9209-4acb-ad60-f68c24323728 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:21.879966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42faee-9209-4acb-ad60-f68c24323728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:21.921787 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:21.947426 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2e42faee-9209-4acb-ad60-f68c24323728 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:21.948654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2e42faee-9209-4acb-ad60-f68c24323728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:22.044775 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:22.330881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8d000-097a-4c53-a4a1-394760dd2793 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:22.332453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cfd8d000-097a-4c53-a4a1-394760dd2793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:22.434594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8d000-097a-4c53-a4a1-394760dd2793 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:22.468575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:22.509925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f947d5-852c-4607-b9ca-4691ee6512ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:22.510528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f947d5-852c-4607-b9ca-4691ee6512ff service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:22.510810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f947d5-852c-4607-b9ca-4691ee6512ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:22.511115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f947d5-852c-4607-b9ca-4691ee6512ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:22.512708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-73f947d5-852c-4607-b9ca-4691ee6512ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:22.620217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f947d5-852c-4607-b9ca-4691ee6512ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:22.621589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f947d5-852c-4607-b9ca-4691ee6512ff service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:22.641638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 725/1443] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:24.706795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149a195-7427-4650-9388-1ac3cb1155bc None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:24.707325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149a195-7427-4650-9388-1ac3cb1155bc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:24.707432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149a195-7427-4650-9388-1ac3cb1155bc None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:24.708712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:24.714227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53295ee5-bc7a-4984-a65d-a618b7ea1b85 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:24.714669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53295ee5-bc7a-4984-a65d-a618b7ea1b85 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:24.714773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53295ee5-bc7a-4984-a65d-a618b7ea1b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:24.780091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-53295ee5-bc7a-4984-a65d-a618b7ea1b85 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:24.781312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-53295ee5-bc7a-4984-a65d-a618b7ea1b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:24.794525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:24.887187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:25.172458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149a195-7427-4650-9388-1ac3cb1155bc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:25.174002 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1149a195-7427-4650-9388-1ac3cb1155bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:25.279264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149a195-7427-4650-9388-1ac3cb1155bc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:25.291791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:25.353938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b93c5-31e3-4c9a-8ad7-ab4c8342f3f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:25.354621 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b93c5-31e3-4c9a-8ad7-ab4c8342f3f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:25.354903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b93c5-31e3-4c9a-8ad7-ab4c8342f3f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:25.355203 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b93c5-31e3-4c9a-8ad7-ab4c8342f3f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:25.356805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-825b93c5-31e3-4c9a-8ad7-ab4c8342f3f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:25.466799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b93c5-31e3-4c9a-8ad7-ab4c8342f3f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:25.468189 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b93c5-31e3-4c9a-8ad7-ab4c8342f3f8 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:25.486618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 720/1446] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:27.723434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facdbb0-b771-43c7-8ffd-1972a01d8106 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:27.723942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facdbb0-b771-43c7-8ffd-1972a01d8106 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:27.724036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facdbb0-b771-43c7-8ffd-1972a01d8106 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:27.725540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:27.731071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad4cac-f069-42a4-a1af-37cc6a21e150 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:27.731462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad4cac-f069-42a4-a1af-37cc6a21e150 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:27.731629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad4cac-f069-42a4-a1af-37cc6a21e150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:27.779393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:27.797604 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c9ad4cac-f069-42a4-a1af-37cc6a21e150 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:27.798836 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c9ad4cac-f069-42a4-a1af-37cc6a21e150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:27.893860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:28.181042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facdbb0-b771-43c7-8ffd-1972a01d8106 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:28.182636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4facdbb0-b771-43c7-8ffd-1972a01d8106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:28.285943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facdbb0-b771-43c7-8ffd-1972a01d8106 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:28.332747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:28.368650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b7ff4-0e82-4147-b1bf-93fd5db1517b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:28.369298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b7ff4-0e82-4147-b1bf-93fd5db1517b service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:28.369576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b7ff4-0e82-4147-b1bf-93fd5db1517b service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:28.369866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b7ff4-0e82-4147-b1bf-93fd5db1517b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:28.371489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1e3b7ff4-0e82-4147-b1bf-93fd5db1517b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:28.480954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b7ff4-0e82-4147-b1bf-93fd5db1517b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:28.482303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b7ff4-0e82-4147-b1bf-93fd5db1517b service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:28.500126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 728/1449] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:30.524710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46138c-70de-44d8-a11c-68601a2ef478 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:30.525245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46138c-70de-44d8-a11c-68601a2ef478 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:30.525245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46138c-70de-44d8-a11c-68601a2ef478 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:30.526580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:30.531713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be19019-6f1f-49a6-ad01-56ff59069a25 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:30.532135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be19019-6f1f-49a6-ad01-56ff59069a25 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:30.532269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be19019-6f1f-49a6-ad01-56ff59069a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:30.599345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7be19019-6f1f-49a6-ad01-56ff59069a25 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:30.600567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7be19019-6f1f-49a6-ad01-56ff59069a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:30.620915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:30.697635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:30.984831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46138c-70de-44d8-a11c-68601a2ef478 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:30.986405 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ad46138c-70de-44d8-a11c-68601a2ef478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:31.089132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46138c-70de-44d8-a11c-68601a2ef478 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:31.108688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:31.161982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce9272-443b-4878-9d07-7296f1d9e8f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:31.162699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce9272-443b-4878-9d07-7296f1d9e8f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:31.162970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce9272-443b-4878-9d07-7296f1d9e8f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:31.163271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce9272-443b-4878-9d07-7296f1d9e8f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:31.164851 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d3ce9272-443b-4878-9d07-7296f1d9e8f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:31.272660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce9272-443b-4878-9d07-7296f1d9e8f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:31.274096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce9272-443b-4878-9d07-7296f1d9e8f9 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:31.293250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 723/1452] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:33.359849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999c30a-258a-4acd-b5d1-cd4536e0d4ca None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:33.360432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999c30a-258a-4acd-b5d1-cd4536e0d4ca None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:33.360432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999c30a-258a-4acd-b5d1-cd4536e0d4ca None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:33.361769 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:33.367309 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8697158-534e-46ce-b79b-bd92f0015659 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:33.367710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8697158-534e-46ce-b79b-bd92f0015659 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:33.367866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8697158-534e-46ce-b79b-bd92f0015659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:33.432548 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:33.435435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e8697158-534e-46ce-b79b-bd92f0015659 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:33.437189 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e8697158-534e-46ce-b79b-bd92f0015659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:33.533288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:33.830015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999c30a-258a-4acd-b5d1-cd4536e0d4ca None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:33.831554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c999c30a-258a-4acd-b5d1-cd4536e0d4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:33.933665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999c30a-258a-4acd-b5d1-cd4536e0d4ca None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:33.982092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:34.006684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa530ee-a1e2-4eff-98c2-634ca1a8a325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:34.007320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa530ee-a1e2-4eff-98c2-634ca1a8a325 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:34.007600 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa530ee-a1e2-4eff-98c2-634ca1a8a325 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:34.007872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa530ee-a1e2-4eff-98c2-634ca1a8a325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:34.009450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0aa530ee-a1e2-4eff-98c2-634ca1a8a325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:34.123364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa530ee-a1e2-4eff-98c2-634ca1a8a325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:34.124795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa530ee-a1e2-4eff-98c2-634ca1a8a325 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:34.143011 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 731/1455] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:36.385159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb69afe-22fc-4aef-8dd9-e95600ed63f5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:36.385679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb69afe-22fc-4aef-8dd9-e95600ed63f5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:36.385801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb69afe-22fc-4aef-8dd9-e95600ed63f5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:36.387076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:36.392391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42c0d-854b-4345-bff5-e2f88aee54e9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:36.392798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42c0d-854b-4345-bff5-e2f88aee54e9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:36.392948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42c0d-854b-4345-bff5-e2f88aee54e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:36.464044 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-78f42c0d-854b-4345-bff5-e2f88aee54e9 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:36.465239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-78f42c0d-854b-4345-bff5-e2f88aee54e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:36.493387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:36.562322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:36.849369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb69afe-22fc-4aef-8dd9-e95600ed63f5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:36.850900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0bb69afe-22fc-4aef-8dd9-e95600ed63f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:36.951342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb69afe-22fc-4aef-8dd9-e95600ed63f5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:36.980284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:37.032621 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec6451-a301-4eed-be1f-c31eab8066ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:37.033228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec6451-a301-4eed-be1f-c31eab8066ac service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:37.033505 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec6451-a301-4eed-be1f-c31eab8066ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:37.033778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec6451-a301-4eed-be1f-c31eab8066ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:37.035408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fcec6451-a301-4eed-be1f-c31eab8066ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:37.142182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec6451-a301-4eed-be1f-c31eab8066ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:37.143579 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec6451-a301-4eed-be1f-c31eab8066ac service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:37.162086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 726/1458] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:39.123500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50dda9-c3da-4abb-9337-d4527f6c4c8e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:39.124096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50dda9-c3da-4abb-9337-d4527f6c4c8e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:39.124096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50dda9-c3da-4abb-9337-d4527f6c4c8e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:39.125383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:39.130656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c6acb-3c1c-493e-ac94-f94ed0add07d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:39.131092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c6acb-3c1c-493e-ac94-f94ed0add07d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:39.131246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c6acb-3c1c-493e-ac94-f94ed0add07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:39.198271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-797c6acb-3c1c-493e-ac94-f94ed0add07d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:39.199756 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-797c6acb-3c1c-493e-ac94-f94ed0add07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:39.208337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:39.295845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:39.584354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50dda9-c3da-4abb-9337-d4527f6c4c8e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:39.585963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4e50dda9-c3da-4abb-9337-d4527f6c4c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:39.689352 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50dda9-c3da-4abb-9337-d4527f6c4c8e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:39.744110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:39.763483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedea38-7d28-438f-8e16-1ebbf99eea35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:39.764083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedea38-7d28-438f-8e16-1ebbf99eea35 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:39.764357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedea38-7d28-438f-8e16-1ebbf99eea35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:39.764625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedea38-7d28-438f-8e16-1ebbf99eea35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:39.766193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9fedea38-7d28-438f-8e16-1ebbf99eea35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:39.873761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedea38-7d28-438f-8e16-1ebbf99eea35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:39.875158 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedea38-7d28-438f-8e16-1ebbf99eea35 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:39.893302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 734/1461] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:41.846456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817430d9-41e3-4113-a6ec-669413729693 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:41.847060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817430d9-41e3-4113-a6ec-669413729693 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:41.847060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817430d9-41e3-4113-a6ec-669413729693 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:41.848564 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:41.854076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9eb66a-dd3e-4c3a-9e4d-46de1b093a88 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:41.854493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9eb66a-dd3e-4c3a-9e4d-46de1b093a88 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:41.854639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9eb66a-dd3e-4c3a-9e4d-46de1b093a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:41.922828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0c9eb66a-dd3e-4c3a-9e4d-46de1b093a88 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:41.924097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0c9eb66a-dd3e-4c3a-9e4d-46de1b093a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:41.963055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:42.022823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:42.314538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817430d9-41e3-4113-a6ec-669413729693 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:42.316134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-817430d9-41e3-4113-a6ec-669413729693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:42.417296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817430d9-41e3-4113-a6ec-669413729693 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:42.454437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:42.492455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e0f79-e011-4161-9d36-765544a2b81f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:42.493468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e0f79-e011-4161-9d36-765544a2b81f service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:42.493993 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e0f79-e011-4161-9d36-765544a2b81f service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:42.494502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e0f79-e011-4161-9d36-765544a2b81f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:42.496293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b35e0f79-e011-4161-9d36-765544a2b81f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:42.612775 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e0f79-e011-4161-9d36-765544a2b81f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:42.614219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e0f79-e011-4161-9d36-765544a2b81f service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:42.631986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 729/1464] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:44.606909 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be49e0-3e72-4f7e-9142-5c4611015c71 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:44.607516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be49e0-3e72-4f7e-9142-5c4611015c71 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:44.607516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be49e0-3e72-4f7e-9142-5c4611015c71 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:44.608813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:44.614781 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721cf71-d3fd-4a30-b6cf-37019ed52611 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:44.615222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721cf71-d3fd-4a30-b6cf-37019ed52611 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:44.615374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721cf71-d3fd-4a30-b6cf-37019ed52611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:44.688584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4721cf71-d3fd-4a30-b6cf-37019ed52611 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:44.690289 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4721cf71-d3fd-4a30-b6cf-37019ed52611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:44.712319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:44.792886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:45.079360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be49e0-3e72-4f7e-9142-5c4611015c71 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:45.080917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c0be49e0-3e72-4f7e-9142-5c4611015c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:45.186395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be49e0-3e72-4f7e-9142-5c4611015c71 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:45.251815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:45.259534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a5c91-b14e-443a-bff4-8a0950457b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:45.260134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a5c91-b14e-443a-bff4-8a0950457b12 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:45.260415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a5c91-b14e-443a-bff4-8a0950457b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:45.260686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a5c91-b14e-443a-bff4-8a0950457b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:45.262259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-427a5c91-b14e-443a-bff4-8a0950457b12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:45.376931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a5c91-b14e-443a-bff4-8a0950457b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:45.378318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a5c91-b14e-443a-bff4-8a0950457b12 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:45.397687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 737/1467] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:47.347218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9204ef-6347-4b51-b269-6fc6cac8b50d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:47.348455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9204ef-6347-4b51-b269-6fc6cac8b50d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:47.348455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9204ef-6347-4b51-b269-6fc6cac8b50d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:47.349669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:55:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:47.355112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fc12e-c21c-4665-9e20-a7e0f4405ad5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:47.355589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fc12e-c21c-4665-9e20-a7e0f4405ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:47.355693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fc12e-c21c-4665-9e20-a7e0f4405ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:47.422987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a12fc12e-c21c-4665-9e20-a7e0f4405ad5 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:47.424233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a12fc12e-c21c-4665-9e20-a7e0f4405ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:47.475981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:47.526842 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:55:47.808454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9204ef-6347-4b51-b269-6fc6cac8b50d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:47.810005 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2e9204ef-6347-4b51-b269-6fc6cac8b50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:47.914139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9204ef-6347-4b51-b269-6fc6cac8b50d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:47.964691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:47.991459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758be0bd-6051-4a7d-83be-02ced5d51141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:47.992064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758be0bd-6051-4a7d-83be-02ced5d51141 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:47.992342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758be0bd-6051-4a7d-83be-02ced5d51141 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:47.992614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758be0bd-6051-4a7d-83be-02ced5d51141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:47.994221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-758be0bd-6051-4a7d-83be-02ced5d51141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:48.107112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758be0bd-6051-4a7d-83be-02ced5d51141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:48.108578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758be0bd-6051-4a7d-83be-02ced5d51141 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:48.126682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 732/1470] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:55:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:55:57.525034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=22 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:58.124431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=66 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:07.596242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c055c9-66f7-43ed-bfa5-e6eee5640c75 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:07.596811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c055c9-66f7-43ed-bfa5-e6eee5640c75 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:07.596811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c055c9-66f7-43ed-bfa5-e6eee5640c75 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:07.598147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 739/1471] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:07.603229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffc28a-ef8a-44ee-a815-238d68afcf96 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:07.603664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffc28a-ef8a-44ee-a815-238d68afcf96 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:07.603766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffc28a-ef8a-44ee-a815-238d68afcf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:07.671364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c8ffc28a-ef8a-44ee-a815-238d68afcf96 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:07.672587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c8ffc28a-ef8a-44ee-a815-238d68afcf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:07.775521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 733/1472] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:56:08.068551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a851c-e2a0-478a-ba3b-df7527e2f86b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:08.068840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a851c-e2a0-478a-ba3b-df7527e2f86b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:08.069119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a851c-e2a0-478a-ba3b-df7527e2f86b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:08.070444 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 740/1473] 10.222.0.86 () {54 vars in 967 bytes} [Mon Aug 29 18:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:08.075557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e959-bc01-4637-ba5e-f353053969fb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:08.075848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e959-bc01-4637-ba5e-f353053969fb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:08.076122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e959-bc01-4637-ba5e-f353053969fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:08.270244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5040e959-bc01-4637-ba5e-f353053969fb None None] MFA Rules not processed for user `9af69888b04c45b09e34d9ff7a0b564f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:08.271502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5040e959-bc01-4637-ba5e-f353053969fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:08.292003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:08.350122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 734/1474] 10.222.0.86 () {58 vars in 1064 bytes} [Mon Aug 29 18:56:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:56:08.357640 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a851c-e2a0-478a-ba3b-df7527e2f86b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:08.359164 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4a3a851c-e2a0-478a-ba3b-df7527e2f86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:08.463322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a851c-e2a0-478a-ba3b-df7527e2f86b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:08.536300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405928f-c2b5-4b01-9668-bacfbab85551 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af69888b04c45b09e34d9ff7a0b564f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:08.536902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405928f-c2b5-4b01-9668-bacfbab85551 service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:08.537181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405928f-c2b5-4b01-9668-bacfbab85551 service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:08.537451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405928f-c2b5-4b01-9668-bacfbab85551 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:08.539056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4405928f-c2b5-4b01-9668-bacfbab85551 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:08.593553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:08.649434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405928f-c2b5-4b01-9668-bacfbab85551 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:08.650806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405928f-c2b5-4b01-9668-bacfbab85551 service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:08.669325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 741/1475] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:56:10.674559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483cda6-610a-4be0-b993-dfb70e8dd79c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:10.675123 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483cda6-610a-4be0-b993-dfb70e8dd79c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:10.675254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483cda6-610a-4be0-b993-dfb70e8dd79c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:10.676944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 735/1476] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:56:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:10.682319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83969f5-9766-4cee-bafb-d0ee53ab102e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:10.682767 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83969f5-9766-4cee-bafb-d0ee53ab102e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:10.682884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83969f5-9766-4cee-bafb-d0ee53ab102e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:10.747871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e83969f5-9766-4cee-bafb-d0ee53ab102e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:10.749058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e83969f5-9766-4cee-bafb-d0ee53ab102e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:10.844005 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 742/1477] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:56:11.116671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483cda6-610a-4be0-b993-dfb70e8dd79c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:11.118207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5483cda6-610a-4be0-b993-dfb70e8dd79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:11.242633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483cda6-610a-4be0-b993-dfb70e8dd79c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:11.291171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:11.334519 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f0d6b-5a6d-47ad-b3cf-338f951f5d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:11.335145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f0d6b-5a6d-47ad-b3cf-338f951f5d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:11.335421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f0d6b-5a6d-47ad-b3cf-338f951f5d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:11.335687 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f0d6b-5a6d-47ad-b3cf-338f951f5d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:11.336312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f0d6b-5a6d-47ad-b3cf-338f951f5d56 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:11.338026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-0e4f0d6b-5a6d-47ad-b3cf-338f951f5d56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 18:56:11.338026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Aug 29 18:56:11.338026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: Aug 29 18:56:11.339215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f0d6b-5a6d-47ad-b3cf-338f951f5d56 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:11.357842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 736/1478] 10.222.0.86 () {56 vars in 1108 bytes} [Mon Aug 29 18:56:11 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 18:56:13.326190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96976619-e8cb-4ad8-b634-e85f4300f599 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:13.326765 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96976619-e8cb-4ad8-b634-e85f4300f599 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:13.326765 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96976619-e8cb-4ad8-b634-e85f4300f599 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:13.328183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 743/1479] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:13.333909 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29651178-8bf9-4f1e-a0af-a378ea580a47 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:13.334335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29651178-8bf9-4f1e-a0af-a378ea580a47 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:13.334443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29651178-8bf9-4f1e-a0af-a378ea580a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:13.400886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-29651178-8bf9-4f1e-a0af-a378ea580a47 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:13.402102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-29651178-8bf9-4f1e-a0af-a378ea580a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:13.498387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 737/1480] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:56:13.793200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96976619-e8cb-4ad8-b634-e85f4300f599 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:13.794719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-96976619-e8cb-4ad8-b634-e85f4300f599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:13.804687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:13.941192 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96976619-e8cb-4ad8-b634-e85f4300f599 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:14.020679 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:14.053948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26825af-a641-43e2-863d-f89430f572df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:14.054559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26825af-a641-43e2-863d-f89430f572df service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:14.054835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26825af-a641-43e2-863d-f89430f572df service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:14.055132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26825af-a641-43e2-863d-f89430f572df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:14.056714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b26825af-a641-43e2-863d-f89430f572df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:14.158495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26825af-a641-43e2-863d-f89430f572df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:14.160121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26825af-a641-43e2-863d-f89430f572df service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:14.180889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 744/1481] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:56:16.307086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235363f-4b8d-4648-b321-624b327268fb None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:16.307679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235363f-4b8d-4648-b321-624b327268fb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:16.307679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235363f-4b8d-4648-b321-624b327268fb None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:16.308962 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 738/1482] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:16.314707 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2f3fc-3985-4e49-939f-677e082bb5eb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:16.315225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2f3fc-3985-4e49-939f-677e082bb5eb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:16.315386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2f3fc-3985-4e49-939f-677e082bb5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:16.381731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cbf2f3fc-3985-4e49-939f-677e082bb5eb None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:16.382948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cbf2f3fc-3985-4e49-939f-677e082bb5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:16.427596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:16.481473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 745/1483] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:56:16.777587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235363f-4b8d-4648-b321-624b327268fb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:16.779161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c235363f-4b8d-4648-b321-624b327268fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:16.802519 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:16.883764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235363f-4b8d-4648-b321-624b327268fb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:16.956619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd6217-f132-4ffa-98ba-312aa6bb085e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:16.957153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd6217-f132-4ffa-98ba-312aa6bb085e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:16.957433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd6217-f132-4ffa-98ba-312aa6bb085e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:16.957707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd6217-f132-4ffa-98ba-312aa6bb085e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:16.959335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-23cd6217-f132-4ffa-98ba-312aa6bb085e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:17.032273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:17.066967 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd6217-f132-4ffa-98ba-312aa6bb085e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:17.068431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd6217-f132-4ffa-98ba-312aa6bb085e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:17.086565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 739/1484] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:56:19.063433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29809782-a127-4368-afc6-1d6408198046 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:19.064496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29809782-a127-4368-afc6-1d6408198046 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:19.064496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29809782-a127-4368-afc6-1d6408198046 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:19.065851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 746/1485] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:56:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:19.071390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c43b-b57c-4e55-9833-bf8ebf7dd6fa None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:19.071831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c43b-b57c-4e55-9833-bf8ebf7dd6fa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:19.071942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c43b-b57c-4e55-9833-bf8ebf7dd6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:19.138894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-aae3c43b-b57c-4e55-9833-bf8ebf7dd6fa None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:19.140147 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-aae3c43b-b57c-4e55-9833-bf8ebf7dd6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:19.239420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 740/1486] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:56:19.524524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29809782-a127-4368-afc6-1d6408198046 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:19.526077 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-29809782-a127-4368-afc6-1d6408198046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:19.628007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29809782-a127-4368-afc6-1d6408198046 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:19.700880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:19.702833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0deda7-3c7a-4198-a39e-b5187e6ec12f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:19.703417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0deda7-3c7a-4198-a39e-b5187e6ec12f service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:19.703884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0deda7-3c7a-4198-a39e-b5187e6ec12f service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:19.704312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0deda7-3c7a-4198-a39e-b5187e6ec12f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:19.706131 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9f0deda7-3c7a-4198-a39e-b5187e6ec12f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:19.813650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0deda7-3c7a-4198-a39e-b5187e6ec12f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:19.815069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0deda7-3c7a-4198-a39e-b5187e6ec12f service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:19.833627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 747/1487] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:56:21.817514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e269786-1c79-46c7-bf31-4c68d3759124 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:21.818072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e269786-1c79-46c7-bf31-4c68d3759124 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:21.818072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e269786-1c79-46c7-bf31-4c68d3759124 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:21.819488 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 741/1488] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:21.824783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f9bb6-0fb1-42a5-bdbe-a9cdd292d2fa None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:21.825188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f9bb6-0fb1-42a5-bdbe-a9cdd292d2fa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:21.825332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f9bb6-0fb1-42a5-bdbe-a9cdd292d2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:21.890706 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-494f9bb6-0fb1-42a5-bdbe-a9cdd292d2fa None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:21.892439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-494f9bb6-0fb1-42a5-bdbe-a9cdd292d2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:21.944503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:21.990096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 748/1489] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:56:22.286750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e269786-1c79-46c7-bf31-4c68d3759124 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:22.288356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4e269786-1c79-46c7-bf31-4c68d3759124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:22.325046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:22.393221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e269786-1c79-46c7-bf31-4c68d3759124 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:22.465534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b31cd8-80e1-462f-86f7-64e3f2f265b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:22.466143 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31cd8-80e1-462f-86f7-64e3f2f265b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:22.466431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31cd8-80e1-462f-86f7-64e3f2f265b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:22.466705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31cd8-80e1-462f-86f7-64e3f2f265b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:22.468341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-08b31cd8-80e1-462f-86f7-64e3f2f265b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:22.548869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:22.578655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b31cd8-80e1-462f-86f7-64e3f2f265b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:22.580125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b31cd8-80e1-462f-86f7-64e3f2f265b1 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:22.598215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 742/1490] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:56:24.677347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f3de8-a1ca-487e-a684-19aa7fe03d02 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:24.677905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f3de8-a1ca-487e-a684-19aa7fe03d02 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:24.677905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f3de8-a1ca-487e-a684-19aa7fe03d02 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:24.679257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 749/1491] 10.222.0.86 () {54 vars in 927 bytes} [Mon Aug 29 18:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:56:24.684633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cf6f6-a856-4f12-842f-512ec049d8cf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:24.685065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cf6f6-a856-4f12-842f-512ec049d8cf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:24.685169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cf6f6-a856-4f12-842f-512ec049d8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:24.750903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-048cf6f6-a856-4f12-842f-512ec049d8cf None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:24.752149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-048cf6f6-a856-4f12-842f-512ec049d8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:24.848265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 743/1492] 10.222.0.86 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:56:25.137438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f3de8-a1ca-487e-a684-19aa7fe03d02 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:25.139069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-626f3de8-a1ca-487e-a684-19aa7fe03d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:25.240316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f3de8-a1ca-487e-a684-19aa7fe03d02 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:25.314057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c34fc-71b3-4cb5-8563-48097615b51b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:25.314660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c34fc-71b3-4cb5-8563-48097615b51b service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:25.314934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c34fc-71b3-4cb5-8563-48097615b51b service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:25.315228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c34fc-71b3-4cb5-8563-48097615b51b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:25.316947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-298c34fc-71b3-4cb5-8563-48097615b51b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:25.326255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:25.426446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c34fc-71b3-4cb5-8563-48097615b51b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:25.427823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c34fc-71b3-4cb5-8563-48097615b51b service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:25.446061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 750/1493] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:56:34.845895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=82 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:35.443808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=50 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:40.806841 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7ef88-2cb6-4680-8ecc-0a5bd922214a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.807458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7ef88-2cb6-4680-8ecc-0a5bd922214a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.807458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7ef88-2cb6-4680-8ecc-0a5bd922214a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:40.884072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-80b7ef88-2cb6-4680-8ecc-0a5bd922214a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:40.885282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-80b7ef88-2cb6-4680-8ecc-0a5bd922214a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:40.981578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 744/1494] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 18:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:57:40.989504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f38812-9092-47de-8012-8de423834134 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.989988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f38812-9092-47de-8012-8de423834134 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.990095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f38812-9092-47de-8012-8de423834134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.065845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-92f38812-9092-47de-8012-8de423834134 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:41.067076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-92f38812-9092-47de-8012-8de423834134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:41.163154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 751/1495] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 18:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:57:41.167754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7ef88-2cb6-4680-8ecc-0a5bd922214a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.169082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-80b7ef88-2cb6-4680-8ecc-0a5bd922214a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:41.180716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7ef88-2cb6-4680-8ecc-0a5bd922214a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.240644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:41.275880 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa0199-e395-43fe-b40e-af19ea4c9220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.276490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0199-e395-43fe-b40e-af19ea4c9220 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.276768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0199-e395-43fe-b40e-af19ea4c9220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.277066 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0199-e395-43fe-b40e-af19ea4c9220 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.277708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa0199-e395-43fe-b40e-af19ea4c9220 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:41.278132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa0199-e395-43fe-b40e-af19ea4c9220 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.279159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-2ffa0199-e395-43fe-b40e-af19ea4c9220 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 18:57:41.279159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Aug 29 18:57:41.279159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: Aug 29 18:57:41.280031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa0199-e395-43fe-b40e-af19ea4c9220 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.290967 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 745/1496] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 18:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:41.299589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f38812-9092-47de-8012-8de423834134 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.309636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f38812-9092-47de-8012-8de423834134 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.369126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:41.403584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349b644-18e1-4b47-9e35-43b2c5b73105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.404194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349b644-18e1-4b47-9e35-43b2c5b73105 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.404470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349b644-18e1-4b47-9e35-43b2c5b73105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.404739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349b644-18e1-4b47-9e35-43b2c5b73105 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.405363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5349b644-18e1-4b47-9e35-43b2c5b73105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.407147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-5349b644-18e1-4b47-9e35-43b2c5b73105 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 18:57:41.407147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Aug 29 18:57:41.407147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: Aug 29 18:57:41.408428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5349b644-18e1-4b47-9e35-43b2c5b73105 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.471617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 752/1497] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 18:57:41 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:57:41.477977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa0199-e395-43fe-b40e-af19ea4c9220 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.480496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2ffa0199-e395-43fe-b40e-af19ea4c9220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:41.518014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa0199-e395-43fe-b40e-af19ea4c9220 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.612087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecddbf6-b6e0-40f2-b7df-039d07a1cb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.612689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecddbf6-b6e0-40f2-b7df-039d07a1cb89 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.612963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecddbf6-b6e0-40f2-b7df-039d07a1cb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.613230 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecddbf6-b6e0-40f2-b7df-039d07a1cb89 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.613842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecddbf6-b6e0-40f2-b7df-039d07a1cb89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.614842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-aecddbf6-b6e0-40f2-b7df-039d07a1cb89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 18:57:41.614842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Aug 29 18:57:41.614842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: Aug 29 18:57:41.616210 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecddbf6-b6e0-40f2-b7df-039d07a1cb89 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.657639 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 746/1498] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 18:57:41 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:57:41.661720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349b644-18e1-4b47-9e35-43b2c5b73105 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.663286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5349b644-18e1-4b47-9e35-43b2c5b73105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:41.727832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349b644-18e1-4b47-9e35-43b2c5b73105 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.815060 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:41.826773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb6118-f52a-485b-be12-51e9f2970d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.827409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb6118-f52a-485b-be12-51e9f2970d31 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.827682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb6118-f52a-485b-be12-51e9f2970d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.827965 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb6118-f52a-485b-be12-51e9f2970d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.828586 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb6118-f52a-485b-be12-51e9f2970d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.829580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-27fb6118-f52a-485b-be12-51e9f2970d31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 18:57:41.829580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Aug 29 18:57:41.829580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: Aug 29 18:57:41.830463 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb6118-f52a-485b-be12-51e9f2970d31 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.843655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 753/1499] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 18:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:57:41.847716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecddbf6-b6e0-40f2-b7df-039d07a1cb89 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.849229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-aecddbf6-b6e0-40f2-b7df-039d07a1cb89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:41.860564 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecddbf6-b6e0-40f2-b7df-039d07a1cb89 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:41.885235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:41.955586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2d9b5-f875-4846-ac37-532898a30715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:41.956189 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2d9b5-f875-4846-ac37-532898a30715 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:41.956466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2d9b5-f875-4846-ac37-532898a30715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:41.956733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2d9b5-f875-4846-ac37-532898a30715 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:41.957366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2d9b5-f875-4846-ac37-532898a30715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:41.958361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-beb2d9b5-f875-4846-ac37-532898a30715 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 18:57:41.958361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Aug 29 18:57:41.958361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: Aug 29 18:57:41.959261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2d9b5-f875-4846-ac37-532898a30715 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:41.971365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 747/1500] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 18:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:57:41.975538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb6118-f52a-485b-be12-51e9f2970d31 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:41.977062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-27fb6118-f52a-485b-be12-51e9f2970d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:42.012368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb6118-f52a-485b-be12-51e9f2970d31 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:42.105900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3e06f-ecb5-4d61-9e71-bcc9593ac8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:42.106475 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e06f-ecb5-4d61-9e71-bcc9593ac8c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:42.106752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e06f-ecb5-4d61-9e71-bcc9593ac8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:42.107052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e06f-ecb5-4d61-9e71-bcc9593ac8c4 admin admin] PATH_INFO: `/v3/projects/a48a072920274f3592c8580b657cc60a/users/ed4be957b4c0410eaeec565ee5fc55b2/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:42.129486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3e06f-ecb5-4d61-9e71-bcc9593ac8c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a48a072920274f3592c8580b657cc60a, user_id=ed4be957b4c0410eaeec565ee5fc55b2, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:42.130536 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-86a3e06f-ecb5-4d61-9e71-bcc9593ac8c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 18:57:42.130536 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Aug 29 18:57:42.130536 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: Aug 29 18:57:42.132529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3e06f-ecb5-4d61-9e71-bcc9593ac8c4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:42.159324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 754/1501] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 18:57:41 2022] PUT /identity/v3/projects/a48a072920274f3592c8580b657cc60a/users/ed4be957b4c0410eaeec565ee5fc55b2/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:57:42.171709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18428de5-a9ac-49b0-8e63-2a930e1a8bb2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:42.172000 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18428de5-a9ac-49b0-8e63-2a930e1a8bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:42.172278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18428de5-a9ac-49b0-8e63-2a930e1a8bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:42.232915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-18428de5-a9ac-49b0-8e63-2a930e1a8bb2 None None] MFA Rules not processed for user `ed4be957b4c0410eaeec565ee5fc55b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:42.234188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-18428de5-a9ac-49b0-8e63-2a930e1a8bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:42.311461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 748/1502] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 18:57:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:57:42.324648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413c175-27fd-4aad-a39f-f27a70203420 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:42.324936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413c175-27fd-4aad-a39f-f27a70203420 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:42.325211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413c175-27fd-4aad-a39f-f27a70203420 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:42.326596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 755/1503] 10.222.0.86 () {54 vars in 964 bytes} [Mon Aug 29 18:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:42.331497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e82059-1edd-493a-8eb7-41fe40c4af0a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:42.331822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e82059-1edd-493a-8eb7-41fe40c4af0a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:42.332093 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e82059-1edd-493a-8eb7-41fe40c4af0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:42.363396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:42.408327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-81e82059-1edd-493a-8eb7-41fe40c4af0a None None] MFA Rules not processed for user `a3030e2c49614aee892c7693da6b73bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:42.409545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-81e82059-1edd-493a-8eb7-41fe40c4af0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:42.484597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 749/1504] 10.222.0.86 () {58 vars in 1061 bytes} [Mon Aug 29 18:57:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:57:42.492520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413c175-27fd-4aad-a39f-f27a70203420 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:42.494046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b413c175-27fd-4aad-a39f-f27a70203420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:42.545046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:42.599556 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413c175-27fd-4aad-a39f-f27a70203420 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:42.672269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab96ff-1382-48e6-a0ab-27b6f804c410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:42.672826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab96ff-1382-48e6-a0ab-27b6f804c410 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:42.673086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab96ff-1382-48e6-a0ab-27b6f804c410 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:42.673357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab96ff-1382-48e6-a0ab-27b6f804c410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:42.675038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-32ab96ff-1382-48e6-a0ab-27b6f804c410 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:42.774374 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab96ff-1382-48e6-a0ab-27b6f804c410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:42.775871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab96ff-1382-48e6-a0ab-27b6f804c410 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:42.780357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:42.796154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 756/1505] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:57:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:57:42.844357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe8ed9-9e94-44b1-ba43-538f2460a162 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:42.844647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe8ed9-9e94-44b1-ba43-538f2460a162 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:42.844922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe8ed9-9e94-44b1-ba43-538f2460a162 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:42.846153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 750/1506] 10.222.0.86 () {52 vars in 830 bytes} [Mon Aug 29 18:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:42.849653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab96ff-1382-48e6-a0ab-27b6f804c410 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:42.850872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-32ab96ff-1382-48e6-a0ab-27b6f804c410 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:42.948818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab96ff-1382-48e6-a0ab-27b6f804c410 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:43.163491 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:43.164189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.164379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.164648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] PATH_INFO: `/v3/users/ed4be957b4c0410eaeec565ee5fc55b2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.181072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4be957b4c0410eaeec565ee5fc55b2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:43.182258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 18:57:43.182258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Aug 29 18:57:43.182258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: Aug 29 18:57:43.184698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:43.196168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:43.276005 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4be957b4c0410eaeec565ee5fc55b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 18:57:43.276451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4be957b4c0410eaeec565ee5fc55b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 18:57:43.299951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4be957b4c0410eaeec565ee5fc55b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 18:57:43.310007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4be957b4c0410eaeec565ee5fc55b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 18:57:43.324095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4be957b4c0410eaeec565ee5fc55b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 18:57:43.330360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 757/1507] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 18:57:42 2022] DELETE /identity/v3/users/ed4be957b4c0410eaeec565ee5fc55b2 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:57:43.334838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27be546-9f02-49ce-a56a-c6767b99fc56 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.335335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27be546-9f02-49ce-a56a-c6767b99fc56 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.335430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27be546-9f02-49ce-a56a-c6767b99fc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.392219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d27be546-9f02-49ce-a56a-c6767b99fc56 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:43.393427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d27be546-9f02-49ce-a56a-c6767b99fc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:43.468689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:43.481708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 751/1508] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 18:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:57:43.492772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:43.494013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:43.600883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34192f7a-734c-4bb5-b893-bb5c8a65e52c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:43.656976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:43.676404 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92c63e-82f2-46a0-8a6c-44a1534803c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:43.677058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92c63e-82f2-46a0-8a6c-44a1534803c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.677337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92c63e-82f2-46a0-8a6c-44a1534803c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.677608 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92c63e-82f2-46a0-8a6c-44a1534803c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.679233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9b92c63e-82f2-46a0-8a6c-44a1534803c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:43.790196 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92c63e-82f2-46a0-8a6c-44a1534803c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:43.791665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92c63e-82f2-46a0-8a6c-44a1534803c4 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:43.810689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 758/1509] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:57:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:57:44.169608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27be546-9f02-49ce-a56a-c6767b99fc56 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:44.170839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d27be546-9f02-49ce-a56a-c6767b99fc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:44.182327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27be546-9f02-49ce-a56a-c6767b99fc56 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:44.288332 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:44.288964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.289275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.289507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] PATH_INFO: `/v3/projects/a48a072920274f3592c8580b657cc60a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.300333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a48a072920274f3592c8580b657cc60a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:44.301359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 18:57:44.301359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Aug 29 18:57:44.301359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: Aug 29 18:57:44.302551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:44.361268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a48a072920274f3592c8580b657cc60a', 'request_id': 'req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 18:57:44.374427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a48a072920274f3592c8580b657cc60a', 'request_id': 'req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 18:57:44.380384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a48a072920274f3592c8580b657cc60a', 'request_id': 'req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 18:57:44.406768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] The token cache is being invalidate because project a48a072920274f3592c8580b657cc60a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 18:57:44.407087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 18:57:44.407513 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-48c3eb59-8e59-4d79-ad3b-4dde5a0555a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 18:57:44.408885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 752/1510] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 18:57:44 2022] DELETE /identity/v3/projects/a48a072920274f3592c8580b657cc60a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:57:53.807742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=62 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.404801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:25.424242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528bae7-825c-4a0f-948b-95a651293557 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.424840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528bae7-825c-4a0f-948b-95a651293557 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.424840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528bae7-825c-4a0f-948b-95a651293557 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.426155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:25.431176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf2e55-f7cc-4cdb-a2c3-ce75d50220aa None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.431648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf2e55-f7cc-4cdb-a2c3-ce75d50220aa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.431754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf2e55-f7cc-4cdb-a2c3-ce75d50220aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.512534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bebf2e55-f7cc-4cdb-a2c3-ce75d50220aa None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:25.513745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bebf2e55-f7cc-4cdb-a2c3-ce75d50220aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:25.615808 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:25.771478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528bae7-825c-4a0f-948b-95a651293557 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:25.774108 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0528bae7-825c-4a0f-948b-95a651293557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:25.894404 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528bae7-825c-4a0f-948b-95a651293557 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:25.900001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27f840-5fef-405a-8168-9841ac0e06a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:25.900622 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27f840-5fef-405a-8168-9841ac0e06a9 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:25.900891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27f840-5fef-405a-8168-9841ac0e06a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:25.901162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27f840-5fef-405a-8168-9841ac0e06a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:25.909694 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27f840-5fef-405a-8168-9841ac0e06a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:25.911298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27f840-5fef-405a-8168-9841ac0e06a9 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:25.920627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 760/1513] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:26.743546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc5fdd-11f1-4bad-9e27-7163153551a1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:26.744035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc5fdd-11f1-4bad-9e27-7163153551a1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:26.744146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc5fdd-11f1-4bad-9e27-7163153551a1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:26.745423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:26.750236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cff8c-f11a-42b7-ab53-fd99a016f2f0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:26.750693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cff8c-f11a-42b7-ab53-fd99a016f2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:26.750789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cff8c-f11a-42b7-ab53-fd99a016f2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:26.770124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-509cff8c-f11a-42b7-ab53-fd99a016f2f0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:26.771355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-509cff8c-f11a-42b7-ab53-fd99a016f2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:26.779166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:26.930617 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc5fdd-11f1-4bad-9e27-7163153551a1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:26.945045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc5fdd-11f1-4bad-9e27-7163153551a1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:26.950235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea30f4-c298-4513-8064-6acf4a3c3ace service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:26.950819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea30f4-c298-4513-8064-6acf4a3c3ace service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:26.951132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea30f4-c298-4513-8064-6acf4a3c3ace service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:26.951405 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea30f4-c298-4513-8064-6acf4a3c3ace service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:26.959280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea30f4-c298-4513-8064-6acf4a3c3ace service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:26.960854 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea30f4-c298-4513-8064-6acf4a3c3ace service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:26.970538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 755/1516] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:29.711372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5ba80-e601-44aa-85a8-d78e32abfcf6 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.711872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5ba80-e601-44aa-85a8-d78e32abfcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.711975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5ba80-e601-44aa-85a8-d78e32abfcf6 None None] PATH_INFO: `/v3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.713198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 762/1517] 10.222.0.89 () {54 vars in 925 bytes} [Mon Aug 29 18:59:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:59:29.720306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799ab1d-8b35-4087-909d-764cb82fa059 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.720718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799ab1d-8b35-4087-909d-764cb82fa059 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.720853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799ab1d-8b35-4087-909d-764cb82fa059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.779915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3799ab1d-8b35-4087-909d-764cb82fa059 None None] MFA Rules not processed for user `27f771f20b734b679d5553d29a8a3884`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:29.781125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3799ab1d-8b35-4087-909d-764cb82fa059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:29.797551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:29.851058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 756/1518] 10.222.0.89 () {58 vars in 1015 bytes} [Mon Aug 29 18:59:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:36.765185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=56 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:38.177672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b5314-3a23-49e7-8574-c76b2ba76581 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.178296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b5314-3a23-49e7-8574-c76b2ba76581 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.178296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b5314-3a23-49e7-8574-c76b2ba76581 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.179501 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 763/1519] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:38.184174 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896a022-582b-4e98-af4f-af5fc0f04ec0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.184599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896a022-582b-4e98-af4f-af5fc0f04ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.184732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896a022-582b-4e98-af4f-af5fc0f04ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.207256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3896a022-582b-4e98-af4f-af5fc0f04ec0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:38.208464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3896a022-582b-4e98-af4f-af5fc0f04ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:38.216969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 757/1520] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:38.371377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b5314-3a23-49e7-8574-c76b2ba76581 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:38.386802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b5314-3a23-49e7-8574-c76b2ba76581 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:38.392205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baf39e-9f94-4727-92d2-fe0c509a6613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:38.392792 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baf39e-9f94-4727-92d2-fe0c509a6613 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:38.393069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baf39e-9f94-4727-92d2-fe0c509a6613 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:38.393337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baf39e-9f94-4727-92d2-fe0c509a6613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:38.401425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9baf39e-9f94-4727-92d2-fe0c509a6613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:38.402986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9baf39e-9f94-4727-92d2-fe0c509a6613 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:38.412308 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 764/1521] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:39.241021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9d603-95e1-42dc-b85b-1bd793992625 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:39.241599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9d603-95e1-42dc-b85b-1bd793992625 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:39.241599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9d603-95e1-42dc-b85b-1bd793992625 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:39.242930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:39.247814 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aff6c-7195-466c-aed2-b0d75ae3ce93 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:39.248232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aff6c-7195-466c-aed2-b0d75ae3ce93 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:39.248369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aff6c-7195-466c-aed2-b0d75ae3ce93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:39.268168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-393aff6c-7195-466c-aed2-b0d75ae3ce93 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:39.269361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-393aff6c-7195-466c-aed2-b0d75ae3ce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:39.277351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:39.428315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9d603-95e1-42dc-b85b-1bd793992625 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:39.442830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9d603-95e1-42dc-b85b-1bd793992625 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:39.448462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fd4a5-0b9c-4f5b-8e8c-7fdec794bb3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:39.449044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fd4a5-0b9c-4f5b-8e8c-7fdec794bb3e service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:39.449316 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fd4a5-0b9c-4f5b-8e8c-7fdec794bb3e service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:39.449582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fd4a5-0b9c-4f5b-8e8c-7fdec794bb3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:39.458105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fd4a5-0b9c-4f5b-8e8c-7fdec794bb3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:39.459701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fd4a5-0b9c-4f5b-8e8c-7fdec794bb3e service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:39.468874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 759/1524] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:40.995240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109c32f-21f4-411e-a290-89dc648afc51 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:40.995798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109c32f-21f4-411e-a290-89dc648afc51 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:40.995798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109c32f-21f4-411e-a290-89dc648afc51 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:40.997055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 766/1525] 10.222.0.86 () {54 vars in 961 bytes} [Mon Aug 29 18:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:41.002595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc03df-a321-4e9d-af7b-e07cc5969891 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.003051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc03df-a321-4e9d-af7b-e07cc5969891 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.003172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc03df-a321-4e9d-af7b-e07cc5969891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.022715 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-dafc03df-a321-4e9d-af7b-e07cc5969891 None None] MFA Rules not processed for user `27f771f20b734b679d5553d29a8a3884`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:41.023928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dafc03df-a321-4e9d-af7b-e07cc5969891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:41.032302 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 760/1526] 10.222.0.86 () {58 vars in 1058 bytes} [Mon Aug 29 18:59:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:41.039820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109c32f-21f4-411e-a290-89dc648afc51 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:41.053083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109c32f-21f4-411e-a290-89dc648afc51 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:41.058035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59796a-95d2-4508-b34f-2aa34a78bfae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:41.058617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59796a-95d2-4508-b34f-2aa34a78bfae service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:41.058890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59796a-95d2-4508-b34f-2aa34a78bfae service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:41.059198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59796a-95d2-4508-b34f-2aa34a78bfae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:41.067701 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59796a-95d2-4508-b34f-2aa34a78bfae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:41.069522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59796a-95d2-4508-b34f-2aa34a78bfae service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:41.079495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 767/1527] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:50.707759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eded21-aa6f-4692-982a-bbc4440ae77b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.708232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eded21-aa6f-4692-982a-bbc4440ae77b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.708328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eded21-aa6f-4692-982a-bbc4440ae77b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.709694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:50.714734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889afd20-a9d9-4bfa-b920-235f18af5a76 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.715148 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889afd20-a9d9-4bfa-b920-235f18af5a76 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.715393 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889afd20-a9d9-4bfa-b920-235f18af5a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.735155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-889afd20-a9d9-4bfa-b920-235f18af5a76 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:50.736586 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-889afd20-a9d9-4bfa-b920-235f18af5a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:50.744824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:50.899902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eded21-aa6f-4692-982a-bbc4440ae77b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:50.914272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eded21-aa6f-4692-982a-bbc4440ae77b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:50.920113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b38a23-5574-4c6f-9f9c-3b916f2ab387 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:50.920698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b38a23-5574-4c6f-9f9c-3b916f2ab387 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.920973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b38a23-5574-4c6f-9f9c-3b916f2ab387 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.921241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b38a23-5574-4c6f-9f9c-3b916f2ab387 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.929213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b38a23-5574-4c6f-9f9c-3b916f2ab387 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:50.930764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b38a23-5574-4c6f-9f9c-3b916f2ab387 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:50.940636 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 762/1530] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:51.787214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcd1d0-b235-49f0-ba26-017fd1fdc225 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:51.787644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcd1d0-b235-49f0-ba26-017fd1fdc225 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:51.787755 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcd1d0-b235-49f0-ba26-017fd1fdc225 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:51.789112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 18:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:51.793965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149dbf0-ff42-4536-9ae8-004099e5a10b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:51.794384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149dbf0-ff42-4536-9ae8-004099e5a10b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:51.794520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149dbf0-ff42-4536-9ae8-004099e5a10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:51.817528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e149dbf0-ff42-4536-9ae8-004099e5a10b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:51.819036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e149dbf0-ff42-4536-9ae8-004099e5a10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:51.829435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 18:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:59:51.991141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcd1d0-b235-49f0-ba26-017fd1fdc225 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:52.006706 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcd1d0-b235-49f0-ba26-017fd1fdc225 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:52.012133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dc144-397b-49b0-bf87-2077cbb546a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:52.012759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dc144-397b-49b0-bf87-2077cbb546a2 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:52.013038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dc144-397b-49b0-bf87-2077cbb546a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:52.013309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dc144-397b-49b0-bf87-2077cbb546a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:52.021391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dc144-397b-49b0-bf87-2077cbb546a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:52.022972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dc144-397b-49b0-bf87-2077cbb546a2 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:52.032474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 770/1533] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:55.944170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd0001-2edb-4201-b136-51aa9e6bff40 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:55.944697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd0001-2edb-4201-b136-51aa9e6bff40 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:55.944697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd0001-2edb-4201-b136-51aa9e6bff40 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:55.945959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:55.951057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c532e4-8184-4250-a4dd-771896975c31 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:55.951458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c532e4-8184-4250-a4dd-771896975c31 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:55.951602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c532e4-8184-4250-a4dd-771896975c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:55.971376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-73c532e4-8184-4250-a4dd-771896975c31 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:55.972552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-73c532e4-8184-4250-a4dd-771896975c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:55.980321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:59:56.262038 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd0001-2edb-4201-b136-51aa9e6bff40 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:56.263794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a9fd0001-2edb-4201-b136-51aa9e6bff40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:56.341673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd0001-2edb-4201-b136-51aa9e6bff40 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:56.346168 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c07740-e3db-4895-a2ae-8970fa2961b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:56.346772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c07740-e3db-4895-a2ae-8970fa2961b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:56.347079 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c07740-e3db-4895-a2ae-8970fa2961b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:56.347354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c07740-e3db-4895-a2ae-8970fa2961b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:56.356502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c07740-e3db-4895-a2ae-8970fa2961b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:56.357917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c07740-e3db-4895-a2ae-8970fa2961b2 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:56.367848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 765/1536] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:58.258067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4afc2a-0449-4c34-bc68-64ebbd319186 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:58.258593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4afc2a-0449-4c34-bc68-64ebbd319186 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:58.258593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4afc2a-0449-4c34-bc68-64ebbd319186 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:58.260024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 18:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:58.265165 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e94c-6f9d-470c-b854-6c9a916b0346 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:58.265557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e94c-6f9d-470c-b854-6c9a916b0346 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:58.265751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e94c-6f9d-470c-b854-6c9a916b0346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:58.285983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8c46e94c-6f9d-470c-b854-6c9a916b0346 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:58.287298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8c46e94c-6f9d-470c-b854-6c9a916b0346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:58.297943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 18:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:58.588309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4afc2a-0449-4c34-bc68-64ebbd319186 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:58.589997 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9a4afc2a-0449-4c34-bc68-64ebbd319186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:58.603392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4afc2a-0449-4c34-bc68-64ebbd319186 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:58.606818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1955d4-e27d-4f3c-b0b0-044f8d415011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:58.607430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1955d4-e27d-4f3c-b0b0-044f8d415011 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:58.607707 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1955d4-e27d-4f3c-b0b0-044f8d415011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:58.607975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1955d4-e27d-4f3c-b0b0-044f8d415011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:58.616723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1955d4-e27d-4f3c-b0b0-044f8d415011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:58.618124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1955d4-e27d-4f3c-b0b0-044f8d415011 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:58.627338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 773/1539] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:02.169933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41be29-e908-4c9e-a4b6-fa47fee1d844 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:02.170530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41be29-e908-4c9e-a4b6-fa47fee1d844 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:02.170530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41be29-e908-4c9e-a4b6-fa47fee1d844 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:02.171898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 767/1540] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:02.177142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e887-80cf-466a-a41a-01ebe8f53398 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:02.177606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e887-80cf-466a-a41a-01ebe8f53398 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:02.177692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e887-80cf-466a-a41a-01ebe8f53398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:02.198141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d193e887-80cf-466a-a41a-01ebe8f53398 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:02.199393 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d193e887-80cf-466a-a41a-01ebe8f53398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:02.208593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 774/1541] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:02.760315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41be29-e908-4c9e-a4b6-fa47fee1d844 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:02.762308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9a41be29-e908-4c9e-a4b6-fa47fee1d844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:02.766446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=83 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:02.842560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41be29-e908-4c9e-a4b6-fa47fee1d844 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:02.846838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb3c73-c311-4298-9014-25126a2b47b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:02.847480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb3c73-c311-4298-9014-25126a2b47b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:02.847763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb3c73-c311-4298-9014-25126a2b47b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:02.848037 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb3c73-c311-4298-9014-25126a2b47b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:02.857085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb3c73-c311-4298-9014-25126a2b47b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:02.858504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb3c73-c311-4298-9014-25126a2b47b6 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:02.868424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 768/1542] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:04.807524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec73e9-8104-4f99-8402-9a6b5b084813 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:04.808076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec73e9-8104-4f99-8402-9a6b5b084813 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:04.808076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec73e9-8104-4f99-8402-9a6b5b084813 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:04.809422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 775/1543] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:04.814673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0805b-400d-4a13-8b91-9667950fb7c6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:04.815110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0805b-400d-4a13-8b91-9667950fb7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:04.815257 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0805b-400d-4a13-8b91-9667950fb7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:04.835178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-85f0805b-400d-4a13-8b91-9667950fb7c6 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:04.836368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-85f0805b-400d-4a13-8b91-9667950fb7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:04.844727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 769/1544] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:05.401560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec73e9-8104-4f99-8402-9a6b5b084813 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:05.403267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e3ec73e9-8104-4f99-8402-9a6b5b084813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:05.416838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec73e9-8104-4f99-8402-9a6b5b084813 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:05.420299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4307b0-3ce3-4528-a37a-eb0dd2534f86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:05.420882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4307b0-3ce3-4528-a37a-eb0dd2534f86 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:05.421154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4307b0-3ce3-4528-a37a-eb0dd2534f86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:05.421418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4307b0-3ce3-4528-a37a-eb0dd2534f86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:05.430042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4307b0-3ce3-4528-a37a-eb0dd2534f86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:05.431482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4307b0-3ce3-4528-a37a-eb0dd2534f86 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:05.440893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 776/1545] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:14.830186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=37 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:15.436822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=42 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:32.560724 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3be76-9c1b-4025-9ee3-31a82c0b4c30 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.564186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3be76-9c1b-4025-9ee3-31a82c0b4c30 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.564186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3be76-9c1b-4025-9ee3-31a82c0b4c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.592307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-19a3be76-9c1b-4025-9ee3-31a82c0b4c30 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:32.593578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-19a3be76-9c1b-4025-9ee3-31a82c0b4c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:32.607085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 770/1546] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:32.616502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cb37c-afd4-4b2d-a45f-e91a409ba405 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.617496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cb37c-afd4-4b2d-a45f-e91a409ba405 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.617982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cb37c-afd4-4b2d-a45f-e91a409ba405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.648009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f52cb37c-afd4-4b2d-a45f-e91a409ba405 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:32.649623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f52cb37c-afd4-4b2d-a45f-e91a409ba405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:32.664872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 777/1547] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:32.669396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3be76-9c1b-4025-9ee3-31a82c0b4c30 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:32.671748 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8066cb-480a-41d1-8535-aaffb9202f58 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.672277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8066cb-480a-41d1-8535-aaffb9202f58 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.673732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8066cb-480a-41d1-8535-aaffb9202f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.684887 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3be76-9c1b-4025-9ee3-31a82c0b4c30 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:32.712981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5e8066cb-480a-41d1-8535-aaffb9202f58 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:32.714453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5e8066cb-480a-41d1-8535-aaffb9202f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:32.715713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75097a33-6ceb-46d6-a776-e926bd5dfd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:32.716352 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75097a33-6ceb-46d6-a776-e926bd5dfd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.716633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75097a33-6ceb-46d6-a776-e926bd5dfd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.716906 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75097a33-6ceb-46d6-a776-e926bd5dfd04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.717561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75097a33-6ceb-46d6-a776-e926bd5dfd04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:01:32.717867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75097a33-6ceb-46d6-a776-e926bd5dfd04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:32.720941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75097a33-6ceb-46d6-a776-e926bd5dfd04 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:32.731041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 778/1548] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:32.741043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59743efd-d91e-45a4-bb74-224927889d1f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.741354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59743efd-d91e-45a4-bb74-224927889d1f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.741594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59743efd-d91e-45a4-bb74-224927889d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.747212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 771/1549] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:01:32.755399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75097a33-6ceb-46d6-a776-e926bd5dfd04 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:32.772977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-59743efd-d91e-45a4-bb74-224927889d1f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:32.773735 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75097a33-6ceb-46d6-a776-e926bd5dfd04 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:32.774170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-59743efd-d91e-45a4-bb74-224927889d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:32.788100 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 779/1550] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:32.789648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f7448-b77b-4a97-8921-d432b2c82df4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.789935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f7448-b77b-4a97-8921-d432b2c82df4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.790205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f7448-b77b-4a97-8921-d432b2c82df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.794691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff07ccd-b9ad-4b88-b36d-547a334cad43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:32.795353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff07ccd-b9ad-4b88-b36d-547a334cad43 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.795638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff07ccd-b9ad-4b88-b36d-547a334cad43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.795914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff07ccd-b9ad-4b88-b36d-547a334cad43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.796553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff07ccd-b9ad-4b88-b36d-547a334cad43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:32.797741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-3ff07ccd-b9ad-4b88-b36d-547a334cad43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:01:32.797741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Aug 29 19:01:32.797741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: Aug 29 19:01:32.799466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff07ccd-b9ad-4b88-b36d-547a334cad43 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:32.831258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-619f7448-b77b-4a97-8921-d432b2c82df4 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:32.832390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-619f7448-b77b-4a97-8921-d432b2c82df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:32.853154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 780/1551] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:32.853810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f7448-b77b-4a97-8921-d432b2c82df4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:32.865995 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f7448-b77b-4a97-8921-d432b2c82df4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:32.867690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 772/1552] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:32 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:32.868969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448064cf-23d7-4f10-8d3e-e9f6a8f50c12 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.869261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448064cf-23d7-4f10-8d3e-e9f6a8f50c12 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.869539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448064cf-23d7-4f10-8d3e-e9f6a8f50c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.904453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-448064cf-23d7-4f10-8d3e-e9f6a8f50c12 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:32.906357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-448064cf-23d7-4f10-8d3e-e9f6a8f50c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:32.947030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eafdcb-4c4a-48ac-8903-3dc109f12fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:32.947030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eafdcb-4c4a-48ac-8903-3dc109f12fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.947030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eafdcb-4c4a-48ac-8903-3dc109f12fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.947030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eafdcb-4c4a-48ac-8903-3dc109f12fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.947517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eafdcb-4c4a-48ac-8903-3dc109f12fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:01:32.947517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eafdcb-4c4a-48ac-8903-3dc109f12fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:32.947517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-04eafdcb-4c4a-48ac-8903-3dc109f12fc1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:01:32.947517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Aug 29 19:01:32.947517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: Aug 29 19:01:32.948329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eafdcb-4c4a-48ac-8903-3dc109f12fc1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:32.963926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 781/1553] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:01:32.965228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da0ae6-4a14-426e-b57a-35b4bab525b0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.965515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da0ae6-4a14-426e-b57a-35b4bab525b0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.965785 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da0ae6-4a14-426e-b57a-35b4bab525b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.988636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-29da0ae6-4a14-426e-b57a-35b4bab525b0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:32.989855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-29da0ae6-4a14-426e-b57a-35b4bab525b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:32.998488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 782/1554] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:32.998823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 773/1555] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:32.999314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da0ae6-4a14-426e-b57a-35b4bab525b0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.000237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0b23b-2999-4c66-90c3-70c1a885b5e3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.000532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0b23b-2999-4c66-90c3-70c1a885b5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.001017 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0b23b-2999-4c66-90c3-70c1a885b5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.010722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da0ae6-4a14-426e-b57a-35b4bab525b0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.015866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465a2df-4a0c-4ccb-afa8-0f3b787e3328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.017175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465a2df-4a0c-4ccb-afa8-0f3b787e3328 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.017465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465a2df-4a0c-4ccb-afa8-0f3b787e3328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.017732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465a2df-4a0c-4ccb-afa8-0f3b787e3328 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.018360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465a2df-4a0c-4ccb-afa8-0f3b787e3328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.019462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-c465a2df-4a0c-4ccb-afa8-0f3b787e3328 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:01:33.019462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Aug 29 19:01:33.019462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: Aug 29 19:01:33.020649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465a2df-4a0c-4ccb-afa8-0f3b787e3328 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.022285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a6a0b23b-2999-4c66-90c3-70c1a885b5e3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:33.023626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a6a0b23b-2999-4c66-90c3-70c1a885b5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:33.033106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 774/1556] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:33.034355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47d34d-9165-4ff9-907d-e91c9142f871 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.034639 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47d34d-9165-4ff9-907d-e91c9142f871 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.034916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47d34d-9165-4ff9-907d-e91c9142f871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.046940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 783/1557] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:32 2022] POST /identity/v3/users => generated 292 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:33.048017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465a2df-4a0c-4ccb-afa8-0f3b787e3328 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.059098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465a2df-4a0c-4ccb-afa8-0f3b787e3328 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.061314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2d47d34d-9165-4ff9-907d-e91c9142f871 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:33.062527 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2d47d34d-9165-4ff9-907d-e91c9142f871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:33.064625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5b616-0306-42ce-acfd-ec921e4dd902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.065323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5b616-0306-42ce-acfd-ec921e4dd902 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.065720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5b616-0306-42ce-acfd-ec921e4dd902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.065992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5b616-0306-42ce-acfd-ec921e4dd902 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.066765 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5b616-0306-42ce-acfd-ec921e4dd902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.069506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5b616-0306-42ce-acfd-ec921e4dd902 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.071098 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 775/1558] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:33.071900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47d34d-9165-4ff9-907d-e91c9142f871 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.090623 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47d34d-9165-4ff9-907d-e91c9142f871 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.097999 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b034021-2f34-452b-9a39-06393d342533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.098628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b034021-2f34-452b-9a39-06393d342533 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.098909 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b034021-2f34-452b-9a39-06393d342533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.099335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b034021-2f34-452b-9a39-06393d342533 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.100068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b034021-2f34-452b-9a39-06393d342533 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:01:33.100754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b034021-2f34-452b-9a39-06393d342533 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.102379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b034021-2f34-452b-9a39-06393d342533 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.111848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 784/1559] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:33.113729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e8245-c6e5-4f7a-a7a4-56243649c8a6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.114231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e8245-c6e5-4f7a-a7a4-56243649c8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.114762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e8245-c6e5-4f7a-a7a4-56243649c8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.115318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 776/1560] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:01:33.116571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027fb94-ef00-4653-9a9e-e13b2f95ae76 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.116928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027fb94-ef00-4653-9a9e-e13b2f95ae76 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.117126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027fb94-ef00-4653-9a9e-e13b2f95ae76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.140555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-783e8245-c6e5-4f7a-a7a4-56243649c8a6 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:33.142961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-783e8245-c6e5-4f7a-a7a4-56243649c8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:33.145275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4027fb94-ef00-4653-9a9e-e13b2f95ae76 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:33.146826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4027fb94-ef00-4653-9a9e-e13b2f95ae76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:33.209824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 785/1561] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:33.210846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e8245-c6e5-4f7a-a7a4-56243649c8a6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.212833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 777/1562] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:33.213899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7b67-9069-4356-a775-9486bf895d49 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.214189 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7b67-9069-4356-a775-9486bf895d49 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.214462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7b67-9069-4356-a775-9486bf895d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.224003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e8245-c6e5-4f7a-a7a4-56243649c8a6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.228652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7021faa-693b-418b-9942-3c6165b1507f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.229248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021faa-693b-418b-9942-3c6165b1507f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.229523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021faa-693b-418b-9942-3c6165b1507f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.229791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021faa-693b-418b-9942-3c6165b1507f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.230422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7021faa-693b-418b-9942-3c6165b1507f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.232050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7021faa-693b-418b-9942-3c6165b1507f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.234578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-057a7b67-9069-4356-a775-9486bf895d49 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:33.235913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-057a7b67-9069-4356-a775-9486bf895d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:33.241992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 786/1563] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:33.242765 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7021faa-693b-418b-9942-3c6165b1507f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.245680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 778/1564] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:33.250055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a7b67-9069-4356-a775-9486bf895d49 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.254489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7021faa-693b-418b-9942-3c6165b1507f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.259102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da4fd8-cd4d-4532-9b83-e4c8ed731f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.259682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da4fd8-cd4d-4532-9b83-e4c8ed731f07 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.259962 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da4fd8-cd4d-4532-9b83-e4c8ed731f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.260232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da4fd8-cd4d-4532-9b83-e4c8ed731f07 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.260680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a7b67-9069-4356-a775-9486bf895d49 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.260847 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da4fd8-cd4d-4532-9b83-e4c8ed731f07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.262736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da4fd8-cd4d-4532-9b83-e4c8ed731f07 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.265279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db000a-5d1d-4e17-b501-b9db01f23cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.265895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db000a-5d1d-4e17-b501-b9db01f23cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.266173 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db000a-5d1d-4e17-b501-b9db01f23cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.266989 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db000a-5d1d-4e17-b501-b9db01f23cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.267661 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db000a-5d1d-4e17-b501-b9db01f23cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.269556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db000a-5d1d-4e17-b501-b9db01f23cf1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.273340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:33.287861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 787/1565] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/users => generated 294 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:33.288050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:33.289777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14658973-92ad-4b26-82fc-3fc7b8f44bde None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.290367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14658973-92ad-4b26-82fc-3fc7b8f44bde None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.290984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14658973-92ad-4b26-82fc-3fc7b8f44bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.303105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 779/1566] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:33.304279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418ab5e-5825-4cbc-8cb4-152244537dcc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.304567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418ab5e-5825-4cbc-8cb4-152244537dcc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.304863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418ab5e-5825-4cbc-8cb4-152244537dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.311225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-14658973-92ad-4b26-82fc-3fc7b8f44bde None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:33.312429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-14658973-92ad-4b26-82fc-3fc7b8f44bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:33.325349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4418ab5e-5825-4cbc-8cb4-152244537dcc None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:33.326551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4418ab5e-5825-4cbc-8cb4-152244537dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:33.374727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 788/1567] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:33.376399 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14658973-92ad-4b26-82fc-3fc7b8f44bde None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.385721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 780/1568] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:33.386574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418ab5e-5825-4cbc-8cb4-152244537dcc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.387320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14658973-92ad-4b26-82fc-3fc7b8f44bde None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.392805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e5377-4612-40a8-8d23-020628710333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.393431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e5377-4612-40a8-8d23-020628710333 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.393818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e5377-4612-40a8-8d23-020628710333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.394047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e5377-4612-40a8-8d23-020628710333 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.394691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e5377-4612-40a8-8d23-020628710333 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:01:33.395027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e5377-4612-40a8-8d23-020628710333 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.396806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e5377-4612-40a8-8d23-020628710333 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.399948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418ab5e-5825-4cbc-8cb4-152244537dcc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.404692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8516fba-50f6-4014-81c9-861cbadf724b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.405287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8516fba-50f6-4014-81c9-861cbadf724b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.405566 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8516fba-50f6-4014-81c9-861cbadf724b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.405832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8516fba-50f6-4014-81c9-861cbadf724b admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.406469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8516fba-50f6-4014-81c9-861cbadf724b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:01:33.406769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8516fba-50f6-4014-81c9-861cbadf724b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.408394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8516fba-50f6-4014-81c9-861cbadf724b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.408970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 789/1569] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:01:33.409858 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e5377-4612-40a8-8d23-020628710333 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.420155 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 781/1570] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:01:33.420961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8516fba-50f6-4014-81c9-861cbadf724b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.425071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e5377-4612-40a8-8d23-020628710333 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.429724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b6943-9ae1-42e7-9a36-b615d26a854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.430318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b6943-9ae1-42e7-9a36-b615d26a854f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.430595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b6943-9ae1-42e7-9a36-b615d26a854f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.430869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b6943-9ae1-42e7-9a36-b615d26a854f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.431571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b6943-9ae1-42e7-9a36-b615d26a854f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.432149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8516fba-50f6-4014-81c9-861cbadf724b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.433149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b6943-9ae1-42e7-9a36-b615d26a854f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.436767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f762c-0466-4c3d-be8b-52a712182907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.437347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f762c-0466-4c3d-be8b-52a712182907 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.437641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f762c-0466-4c3d-be8b-52a712182907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.437895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f762c-0466-4c3d-be8b-52a712182907 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.439141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f762c-0466-4c3d-be8b-52a712182907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:01:33.439443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f762c-0466-4c3d-be8b-52a712182907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.440984 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f762c-0466-4c3d-be8b-52a712182907 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.444424 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 790/1571] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:33.445213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b6943-9ae1-42e7-9a36-b615d26a854f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.453205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 782/1572] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:01:33.455058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f762c-0466-4c3d-be8b-52a712182907 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.455617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b6943-9ae1-42e7-9a36-b615d26a854f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.460293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b350a9-7f8c-49d9-b393-c1549dd2d3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.460755 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b350a9-7f8c-49d9-b393-c1549dd2d3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.461031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b350a9-7f8c-49d9-b393-c1549dd2d3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.461302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b350a9-7f8c-49d9-b393-c1549dd2d3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.462438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b350a9-7f8c-49d9-b393-c1549dd2d3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.463550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b350a9-7f8c-49d9-b393-c1549dd2d3e2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.465944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f762c-0466-4c3d-be8b-52a712182907 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.470627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69832964-4551-4988-8618-7244d24b0e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.471271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69832964-4551-4988-8618-7244d24b0e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.471544 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69832964-4551-4988-8618-7244d24b0e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.471814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69832964-4551-4988-8618-7244d24b0e5d admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.472454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69832964-4551-4988-8618-7244d24b0e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.473876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 791/1573] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:33.474327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69832964-4551-4988-8618-7244d24b0e5d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.475073 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4c4a5-fecc-42d8-936f-804556836e74 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.475359 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4c4a5-fecc-42d8-936f-804556836e74 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.475633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4c4a5-fecc-42d8-936f-804556836e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.494577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e9c4c4a5-fecc-42d8-936f-804556836e74 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:33.495804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e9c4c4a5-fecc-42d8-936f-804556836e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:33.496319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 783/1574] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:33.497552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173ba33-9e9b-4a17-b6a4-189b23b5ca00 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.497843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173ba33-9e9b-4a17-b6a4-189b23b5ca00 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.498143 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173ba33-9e9b-4a17-b6a4-189b23b5ca00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.505699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 792/1575] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:33.506469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4c4a5-fecc-42d8-936f-804556836e74 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.515865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4c4a5-fecc-42d8-936f-804556836e74 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.520998 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba21971-bd58-4081-a239-07378cff1df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.521183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0173ba33-9e9b-4a17-b6a4-189b23b5ca00 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:33.521588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21971-bd58-4081-a239-07378cff1df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.521866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21971-bd58-4081-a239-07378cff1df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.522148 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21971-bd58-4081-a239-07378cff1df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.522285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0173ba33-9e9b-4a17-b6a4-189b23b5ca00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:33.522757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba21971-bd58-4081-a239-07378cff1df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.524659 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba21971-bd58-4081-a239-07378cff1df1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.530243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 784/1576] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:33.531145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173ba33-9e9b-4a17-b6a4-189b23b5ca00 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.540439 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173ba33-9e9b-4a17-b6a4-189b23b5ca00 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.545535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661a04c-31a1-4e0d-9050-7634fbbf6263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.546158 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661a04c-31a1-4e0d-9050-7634fbbf6263 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.546434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661a04c-31a1-4e0d-9050-7634fbbf6263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.546881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661a04c-31a1-4e0d-9050-7634fbbf6263 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.547351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661a04c-31a1-4e0d-9050-7634fbbf6263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.549228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661a04c-31a1-4e0d-9050-7634fbbf6263 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.552668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 793/1577] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:33.553614 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba21971-bd58-4081-a239-07378cff1df1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.580092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 785/1578] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:33.580807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661a04c-31a1-4e0d-9050-7634fbbf6263 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.617074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba21971-bd58-4081-a239-07378cff1df1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.643687 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661a04c-31a1-4e0d-9050-7634fbbf6263 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.648245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb112d-d03c-4ef5-97aa-5207e1a2aa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.648847 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb112d-d03c-4ef5-97aa-5207e1a2aa53 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.649128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb112d-d03c-4ef5-97aa-5207e1a2aa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.649404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb112d-d03c-4ef5-97aa-5207e1a2aa53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.649565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:33.650032 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb112d-d03c-4ef5-97aa-5207e1a2aa53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.652108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb112d-d03c-4ef5-97aa-5207e1a2aa53 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.675059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15846c-9797-4776-8814-97bbfeb2e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.675872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15846c-9797-4776-8814-97bbfeb2e9ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.676161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15846c-9797-4776-8814-97bbfeb2e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.676438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15846c-9797-4776-8814-97bbfeb2e9ad admin admin] PATH_INFO: `/v3/projects/90b1cb2f10754e4d90caf3e3030c1eb4/users/cfe7c4e4e56e44d0aa562be2195bfdec/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.683978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 786/1579] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:33.687911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb112d-d03c-4ef5-97aa-5207e1a2aa53 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.695496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15846c-9797-4776-8814-97bbfeb2e9ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b1cb2f10754e4d90caf3e3030c1eb4, user_id=cfe7c4e4e56e44d0aa562be2195bfdec, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.698174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15846c-9797-4776-8814-97bbfeb2e9ad admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.698637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:33.711705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 794/1580] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:33 2022] PUT /identity/v3/projects/90b1cb2f10754e4d90caf3e3030c1eb4/users/cfe7c4e4e56e44d0aa562be2195bfdec/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:33.712604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15846c-9797-4776-8814-97bbfeb2e9ad admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.764472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb112d-d03c-4ef5-97aa-5207e1a2aa53 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.779172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15846c-9797-4776-8814-97bbfeb2e9ad admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.783813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7159069-303a-4f0f-a920-5de05da26daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.784419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7159069-303a-4f0f-a920-5de05da26daa admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.784695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7159069-303a-4f0f-a920-5de05da26daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.784965 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7159069-303a-4f0f-a920-5de05da26daa admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.785593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7159069-303a-4f0f-a920-5de05da26daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.787203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7159069-303a-4f0f-a920-5de05da26daa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.796882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 795/1581] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:33.798266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7159069-303a-4f0f-a920-5de05da26daa admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.807859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7159069-303a-4f0f-a920-5de05da26daa admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.812613 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecda16-76b0-4d32-94a4-3607da47dc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.813231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecda16-76b0-4d32-94a4-3607da47dc57 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.814071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecda16-76b0-4d32-94a4-3607da47dc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.814299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecda16-76b0-4d32-94a4-3607da47dc57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.814928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaecda16-76b0-4d32-94a4-3607da47dc57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:01:33.815261 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaecda16-76b0-4d32-94a4-3607da47dc57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.816830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaecda16-76b0-4d32-94a4-3607da47dc57 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.823432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389c3e2-24f2-40b1-9d08-731ee7fb0c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.824046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389c3e2-24f2-40b1-9d08-731ee7fb0c38 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.824329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389c3e2-24f2-40b1-9d08-731ee7fb0c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.824604 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389c3e2-24f2-40b1-9d08-731ee7fb0c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.825279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389c3e2-24f2-40b1-9d08-731ee7fb0c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.826898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389c3e2-24f2-40b1-9d08-731ee7fb0c38 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.828872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 796/1582] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:01:33.829740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecda16-76b0-4d32-94a4-3607da47dc57 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.839323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecda16-76b0-4d32-94a4-3607da47dc57 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.846200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 787/1583] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:33.846200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389c3e2-24f2-40b1-9d08-731ee7fb0c38 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.846754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60af93-beeb-4327-b493-4db1cddd3450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.846754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60af93-beeb-4327-b493-4db1cddd3450 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.846754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60af93-beeb-4327-b493-4db1cddd3450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.846754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60af93-beeb-4327-b493-4db1cddd3450 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.847173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60af93-beeb-4327-b493-4db1cddd3450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:01:33.847173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60af93-beeb-4327-b493-4db1cddd3450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.847913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60af93-beeb-4327-b493-4db1cddd3450 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.854433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389c3e2-24f2-40b1-9d08-731ee7fb0c38 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.859265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18db835-0ed5-4f33-80e6-7eab7655b7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.859851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 797/1584] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:01:33.859990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18db835-0ed5-4f33-80e6-7eab7655b7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.860733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18db835-0ed5-4f33-80e6-7eab7655b7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.861004 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18db835-0ed5-4f33-80e6-7eab7655b7d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.861222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60af93-beeb-4327-b493-4db1cddd3450 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.861765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18db835-0ed5-4f33-80e6-7eab7655b7d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.863751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18db835-0ed5-4f33-80e6-7eab7655b7d7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.877200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60af93-beeb-4327-b493-4db1cddd3450 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.882413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ff10f-3450-4c02-826a-6731a71fea3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.883063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ff10f-3450-4c02-826a-6731a71fea3c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.883500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ff10f-3450-4c02-826a-6731a71fea3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.883779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ff10f-3450-4c02-826a-6731a71fea3c admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.884436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ff10f-3450-4c02-826a-6731a71fea3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.886325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ff10f-3450-4c02-826a-6731a71fea3c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.889876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 788/1585] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:33.889876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18db835-0ed5-4f33-80e6-7eab7655b7d7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.901144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18db835-0ed5-4f33-80e6-7eab7655b7d7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.907681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523d85f-0e62-404a-aa23-61a4e4adfafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.908597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523d85f-0e62-404a-aa23-61a4e4adfafd admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.908945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523d85f-0e62-404a-aa23-61a4e4adfafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.909219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523d85f-0e62-404a-aa23-61a4e4adfafd admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.909839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523d85f-0e62-404a-aa23-61a4e4adfafd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.911780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523d85f-0e62-404a-aa23-61a4e4adfafd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.912363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 798/1586] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/users => generated 288 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:33.913614 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f2782-9ccf-48aa-8ee5-d97decf16ce9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.913906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f2782-9ccf-48aa-8ee5-d97decf16ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.914183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f2782-9ccf-48aa-8ee5-d97decf16ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.939410 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 789/1587] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/users => generated 282 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:33.940251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523d85f-0e62-404a-aa23-61a4e4adfafd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.951186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-803f2782-9ccf-48aa-8ee5-d97decf16ce9 None None] MFA Rules not processed for user `cfe7c4e4e56e44d0aa562be2195bfdec`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:33.951473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523d85f-0e62-404a-aa23-61a4e4adfafd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.952409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-803f2782-9ccf-48aa-8ee5-d97decf16ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:33.956399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612c93c-049d-460d-b00d-3fe6f3cf08ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.957069 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612c93c-049d-460d-b00d-3fe6f3cf08ac admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.957474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612c93c-049d-460d-b00d-3fe6f3cf08ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.957822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612c93c-049d-460d-b00d-3fe6f3cf08ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.958456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612c93c-049d-460d-b00d-3fe6f3cf08ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.960076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612c93c-049d-460d-b00d-3fe6f3cf08ac admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:33.970753 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 790/1588] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:33.971468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612c93c-049d-460d-b00d-3fe6f3cf08ac admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:33.981129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612c93c-049d-460d-b00d-3fe6f3cf08ac admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:33.987234 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99631600-3006-479d-afc7-c20ab538098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:33.987817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99631600-3006-479d-afc7-c20ab538098a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.988096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99631600-3006-479d-afc7-c20ab538098a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.988371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99631600-3006-479d-afc7-c20ab538098a admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.988988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99631600-3006-479d-afc7-c20ab538098a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:33.990861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99631600-3006-479d-afc7-c20ab538098a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.017487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 791/1589] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:34.018122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99631600-3006-479d-afc7-c20ab538098a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.022198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 799/1590] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:34.023183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f2782-9ccf-48aa-8ee5-d97decf16ce9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.032892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:34.038990 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f2782-9ccf-48aa-8ee5-d97decf16ce9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.072163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:34.089602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99631600-3006-479d-afc7-c20ab538098a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.094205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a427a6-5a57-4d86-9c6b-7b782eeced5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.095470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a427a6-5a57-4d86-9c6b-7b782eeced5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.095751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a427a6-5a57-4d86-9c6b-7b782eeced5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.096025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a427a6-5a57-4d86-9c6b-7b782eeced5a admin admin] PATH_INFO: `/v3/projects/b3171bab0ea547d9acc3559f449221eb/users/dd013ba83e944ecfa1a2d0a8a0d55101/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.104281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05923570-a453-4050-9ff4-3d2289315c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.104891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05923570-a453-4050-9ff4-3d2289315c99 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.105173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05923570-a453-4050-9ff4-3d2289315c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.105449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05923570-a453-4050-9ff4-3d2289315c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.106105 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05923570-a453-4050-9ff4-3d2289315c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.108056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05923570-a453-4050-9ff4-3d2289315c99 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.111194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a427a6-5a57-4d86-9c6b-7b782eeced5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3171bab0ea547d9acc3559f449221eb, user_id=dd013ba83e944ecfa1a2d0a8a0d55101, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.112213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-e3a427a6-5a57-4d86-9c6b-7b782eeced5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:01:34.112213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Aug 29 19:01:34.112213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: Aug 29 19:01:34.114262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a427a6-5a57-4d86-9c6b-7b782eeced5a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.128893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 792/1591] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:34 2022] PUT /identity/v3/projects/b3171bab0ea547d9acc3559f449221eb/users/dd013ba83e944ecfa1a2d0a8a0d55101/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:34.133339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a427a6-5a57-4d86-9c6b-7b782eeced5a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.135872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 800/1592] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:34 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:34.136583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05923570-a453-4050-9ff4-3d2289315c99 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.200721 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05923570-a453-4050-9ff4-3d2289315c99 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.201264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a427a6-5a57-4d86-9c6b-7b782eeced5a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.206336 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af55eb-3635-4aee-84a1-9f4a7bc774c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.206980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af55eb-3635-4aee-84a1-9f4a7bc774c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.207248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303f2b2-76ab-4df9-aaa7-de6cccae0df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.207391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af55eb-3635-4aee-84a1-9f4a7bc774c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.207549 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af55eb-3635-4aee-84a1-9f4a7bc774c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.207931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303f2b2-76ab-4df9-aaa7-de6cccae0df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.208220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303f2b2-76ab-4df9-aaa7-de6cccae0df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.208330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af55eb-3635-4aee-84a1-9f4a7bc774c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.208497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303f2b2-76ab-4df9-aaa7-de6cccae0df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.209148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303f2b2-76ab-4df9-aaa7-de6cccae0df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.211868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303f2b2-76ab-4df9-aaa7-de6cccae0df6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.214605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af55eb-3635-4aee-84a1-9f4a7bc774c1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.231733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 793/1593] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:34.231733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303f2b2-76ab-4df9-aaa7-de6cccae0df6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.232051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 801/1594] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:34.232051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af55eb-3635-4aee-84a1-9f4a7bc774c1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.240114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af55eb-3635-4aee-84a1-9f4a7bc774c1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.242872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303f2b2-76ab-4df9-aaa7-de6cccae0df6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.244736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc41df-f7fc-4341-b6a4-ac26412dadb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.245377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc41df-f7fc-4341-b6a4-ac26412dadb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.245656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc41df-f7fc-4341-b6a4-ac26412dadb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.245934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc41df-f7fc-4341-b6a4-ac26412dadb1 admin admin] PATH_INFO: `/v3/projects/4c6272dcd88f47abb7a514070aaaccd3/users/5598718e51df4a7c92ef828de3003dc7/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.247829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58036b62-5487-4c2b-a81e-965a43296069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.248448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58036b62-5487-4c2b-a81e-965a43296069 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.248746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58036b62-5487-4c2b-a81e-965a43296069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.249021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58036b62-5487-4c2b-a81e-965a43296069 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.249680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58036b62-5487-4c2b-a81e-965a43296069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.251343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58036b62-5487-4c2b-a81e-965a43296069 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.260687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc41df-f7fc-4341-b6a4-ac26412dadb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6272dcd88f47abb7a514070aaaccd3, user_id=5598718e51df4a7c92ef828de3003dc7, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.261317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 794/1595] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:34.261929 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58036b62-5487-4c2b-a81e-965a43296069 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.263372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc41df-f7fc-4341-b6a4-ac26412dadb1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.271931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58036b62-5487-4c2b-a81e-965a43296069 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.277885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 802/1596] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:34 2022] PUT /identity/v3/projects/4c6272dcd88f47abb7a514070aaaccd3/users/5598718e51df4a7c92ef828de3003dc7/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:34.279260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740b8df-ffc7-4ddc-b263-e40d83a0f78c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.279646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740b8df-ffc7-4ddc-b263-e40d83a0f78c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.280024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740b8df-ffc7-4ddc-b263-e40d83a0f78c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.282508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 803/1597] 10.222.0.86 () {54 vars in 962 bytes} [Mon Aug 29 19:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:34.283472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975c16-8eb3-462d-81be-3f70a8d1c37a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.283757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975c16-8eb3-462d-81be-3f70a8d1c37a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.284141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975c16-8eb3-462d-81be-3f70a8d1c37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.304076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f1975c16-8eb3-462d-81be-3f70a8d1c37a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:34.305430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f1975c16-8eb3-462d-81be-3f70a8d1c37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:34.332636 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39007093-4e89-41b5-8949-55f244c6d94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.333253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39007093-4e89-41b5-8949-55f244c6d94e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.333532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39007093-4e89-41b5-8949-55f244c6d94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.333811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39007093-4e89-41b5-8949-55f244c6d94e admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.334440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39007093-4e89-41b5-8949-55f244c6d94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.336395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39007093-4e89-41b5-8949-55f244c6d94e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.357775 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:34.358948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 795/1598] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:34 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:34.359880 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39007093-4e89-41b5-8949-55f244c6d94e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.370266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39007093-4e89-41b5-8949-55f244c6d94e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.371357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 804/1599] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:34.372081 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1975c16-8eb3-462d-81be-3f70a8d1c37a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.378294 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa0a12-344d-4120-9a65-11dcb192fab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.378911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0a12-344d-4120-9a65-11dcb192fab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.379219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0a12-344d-4120-9a65-11dcb192fab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.379496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0a12-344d-4120-9a65-11dcb192fab7 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.380209 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa0a12-344d-4120-9a65-11dcb192fab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.382057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa0a12-344d-4120-9a65-11dcb192fab7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.382455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1975c16-8eb3-462d-81be-3f70a8d1c37a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.387232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28acb8d-16bf-489c-be7f-c3ada26f10dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.387831 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28acb8d-16bf-489c-be7f-c3ada26f10dd admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.388111 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28acb8d-16bf-489c-be7f-c3ada26f10dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.388414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28acb8d-16bf-489c-be7f-c3ada26f10dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.389055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28acb8d-16bf-489c-be7f-c3ada26f10dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.390700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28acb8d-16bf-489c-be7f-c3ada26f10dd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.405233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 805/1600] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:34.405233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28acb8d-16bf-489c-be7f-c3ada26f10dd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.408632 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 796/1601] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:34 2022] POST /identity/v3/users => generated 303 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:34.409679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa0a12-344d-4120-9a65-11dcb192fab7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.418173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28acb8d-16bf-489c-be7f-c3ada26f10dd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.419295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa0a12-344d-4120-9a65-11dcb192fab7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.422955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbd50b-c1c9-4f40-9290-bb708d962e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.423577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd50b-c1c9-4f40-9290-bb708d962e4e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.423777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aefaad-77dc-4739-9756-0fe344564196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.423895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd50b-c1c9-4f40-9290-bb708d962e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.424121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd50b-c1c9-4f40-9290-bb708d962e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.424350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aefaad-77dc-4739-9756-0fe344564196 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.424630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aefaad-77dc-4739-9756-0fe344564196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.424735 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbd50b-c1c9-4f40-9290-bb708d962e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.424897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aefaad-77dc-4739-9756-0fe344564196 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.425522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aefaad-77dc-4739-9756-0fe344564196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.426316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbd50b-c1c9-4f40-9290-bb708d962e4e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.427108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aefaad-77dc-4739-9756-0fe344564196 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.436764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 806/1602] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:34.438035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 797/1603] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:34.440976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa57595-6377-448f-9d21-5a428127bb8b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.441165 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a91b5-4f6b-48bf-a8bc-e437634a2c14 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.441461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a91b5-4f6b-48bf-a8bc-e437634a2c14 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.441607 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa57595-6377-448f-9d21-5a428127bb8b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.441739 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a91b5-4f6b-48bf-a8bc-e437634a2c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.441888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa57595-6377-448f-9d21-5a428127bb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.462248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3fa57595-6377-448f-9d21-5a428127bb8b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:34.463633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3fa57595-6377-448f-9d21-5a428127bb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:34.466057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:34.473201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 807/1604] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:34.474079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa57595-6377-448f-9d21-5a428127bb8b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.483789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa57595-6377-448f-9d21-5a428127bb8b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.485954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-129a91b5-4f6b-48bf-a8bc-e437634a2c14 None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:34.487184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-129a91b5-4f6b-48bf-a8bc-e437634a2c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:34.489022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bbf87-07c8-4c06-96ca-f8345355ef9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.489639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bbf87-07c8-4c06-96ca-f8345355ef9a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.489941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bbf87-07c8-4c06-96ca-f8345355ef9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.490187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bbf87-07c8-4c06-96ca-f8345355ef9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.490824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2bbf87-07c8-4c06-96ca-f8345355ef9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.492647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2bbf87-07c8-4c06-96ca-f8345355ef9a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.503117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 808/1605] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:34.503921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bbf87-07c8-4c06-96ca-f8345355ef9a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.553617 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 798/1606] 10.222.0.86 () {58 vars in 1059 bytes} [Mon Aug 29 19:01:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:34.557680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bbf87-07c8-4c06-96ca-f8345355ef9a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.559895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a91b5-4f6b-48bf-a8bc-e437634a2c14 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.561328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb148e82-40ee-4226-9784-e2b168243a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.561935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148e82-40ee-4226-9784-e2b168243a82 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.562217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148e82-40ee-4226-9784-e2b168243a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.562488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148e82-40ee-4226-9784-e2b168243a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.572019 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a91b5-4f6b-48bf-a8bc-e437634a2c14 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.572363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb148e82-40ee-4226-9784-e2b168243a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.573746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb148e82-40ee-4226-9784-e2b168243a82 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.577462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e9f92-790f-4d7c-a0b9-d4a55eae8f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.578370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e9f92-790f-4d7c-a0b9-d4a55eae8f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.578675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e9f92-790f-4d7c-a0b9-d4a55eae8f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.579680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e9f92-790f-4d7c-a0b9-d4a55eae8f01 admin admin] PATH_INFO: `/v3/projects/12c146a9d54f4ed0a4e8a4a835d6367f/users/b114570010d94eea955b7b3f1fa97c0a/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.585507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 809/1607] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:34.586350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb148e82-40ee-4226-9784-e2b168243a82 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.596824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb148e82-40ee-4226-9784-e2b168243a82 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.597567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e9f92-790f-4d7c-a0b9-d4a55eae8f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c146a9d54f4ed0a4e8a4a835d6367f, user_id=b114570010d94eea955b7b3f1fa97c0a, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.600792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e9f92-790f-4d7c-a0b9-d4a55eae8f01 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.602614 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae1d95-ffe2-4f2a-8bb1-44699a0a85f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.603240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae1d95-ffe2-4f2a-8bb1-44699a0a85f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.603519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae1d95-ffe2-4f2a-8bb1-44699a0a85f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.603791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae1d95-ffe2-4f2a-8bb1-44699a0a85f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.604421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae1d95-ffe2-4f2a-8bb1-44699a0a85f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.606009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae1d95-ffe2-4f2a-8bb1-44699a0a85f5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.620329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 810/1608] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:34.620329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae1d95-ffe2-4f2a-8bb1-44699a0a85f5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.621044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 799/1609] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:34 2022] PUT /identity/v3/projects/12c146a9d54f4ed0a4e8a4a835d6367f/users/b114570010d94eea955b7b3f1fa97c0a/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:34.621911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e9f92-790f-4d7c-a0b9-d4a55eae8f01 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.628208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae1d95-ffe2-4f2a-8bb1-44699a0a85f5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.695843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4048-89ae-4cc0-9b47-ad0ef62fd5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.696671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4048-89ae-4cc0-9b47-ad0ef62fd5ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.696951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4048-89ae-4cc0-9b47-ad0ef62fd5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.698024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4048-89ae-4cc0-9b47-ad0ef62fd5ba admin admin] PATH_INFO: `/v3/projects/3cb291f3145d45feb6e34d6674ce29a9/users/31cb3c3b3b0a451980b09eecbb708a65/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.710683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e9f92-790f-4d7c-a0b9-d4a55eae8f01 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.714204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f4048-89ae-4cc0-9b47-ad0ef62fd5ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb291f3145d45feb6e34d6674ce29a9, user_id=31cb3c3b3b0a451980b09eecbb708a65, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.716101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a97d6c-1c1b-4f13-9c8a-90146a6fc160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.716764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a97d6c-1c1b-4f13-9c8a-90146a6fc160 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.717048 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a97d6c-1c1b-4f13-9c8a-90146a6fc160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.717577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a97d6c-1c1b-4f13-9c8a-90146a6fc160 admin admin] PATH_INFO: `/v3/projects/4c10f21807bb4d53a03d1b3e11f365fe/users/c7dd8b82a5284a9ea576a6ae27bc5664/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.717701 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f4048-89ae-4cc0-9b47-ad0ef62fd5ba admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.734098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 811/1610] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:34 2022] PUT /identity/v3/projects/3cb291f3145d45feb6e34d6674ce29a9/users/31cb3c3b3b0a451980b09eecbb708a65/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:34.734424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a97d6c-1c1b-4f13-9c8a-90146a6fc160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c10f21807bb4d53a03d1b3e11f365fe, user_id=c7dd8b82a5284a9ea576a6ae27bc5664, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.737361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a97d6c-1c1b-4f13-9c8a-90146a6fc160 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.740697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4048-89ae-4cc0-9b47-ad0ef62fd5ba admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.756080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 800/1611] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:34 2022] PUT /identity/v3/projects/4c10f21807bb4d53a03d1b3e11f365fe/users/c7dd8b82a5284a9ea576a6ae27bc5664/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:34.756914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a97d6c-1c1b-4f13-9c8a-90146a6fc160 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.800611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:34.824096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4048-89ae-4cc0-9b47-ad0ef62fd5ba admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.826225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:34.841772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a97d6c-1c1b-4f13-9c8a-90146a6fc160 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.848722 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a46be5-bc1c-490b-88ba-df9815e58c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.849526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46be5-bc1c-490b-88ba-df9815e58c9b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.849945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46be5-bc1c-490b-88ba-df9815e58c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.850705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46be5-bc1c-490b-88ba-df9815e58c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.851855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a46be5-bc1c-490b-88ba-df9815e58c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.854163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a46be5-bc1c-490b-88ba-df9815e58c9b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.865266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 801/1612] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:34.866117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a46be5-bc1c-490b-88ba-df9815e58c9b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.886605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49686331-42d1-40bc-8198-adbf4de40b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.887445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49686331-42d1-40bc-8198-adbf4de40b99 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.887729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49686331-42d1-40bc-8198-adbf4de40b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.888003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49686331-42d1-40bc-8198-adbf4de40b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.901406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49686331-42d1-40bc-8198-adbf4de40b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.903539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49686331-42d1-40bc-8198-adbf4de40b99 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.914448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 812/1613] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:34.915300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49686331-42d1-40bc-8198-adbf4de40b99 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.924888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49686331-42d1-40bc-8198-adbf4de40b99 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.931128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e056e2a-a7d8-4572-9bb8-a6dcc0587e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.931725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e056e2a-a7d8-4572-9bb8-a6dcc0587e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.932003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e056e2a-a7d8-4572-9bb8-a6dcc0587e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.932268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e056e2a-a7d8-4572-9bb8-a6dcc0587e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.933068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e056e2a-a7d8-4572-9bb8-a6dcc0587e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.934713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e056e2a-a7d8-4572-9bb8-a6dcc0587e99 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.946652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 813/1614] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:34.947998 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a62fd-7579-4a34-b2cb-7fba053e556a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.948288 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a62fd-7579-4a34-b2cb-7fba053e556a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.948563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a62fd-7579-4a34-b2cb-7fba053e556a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.955325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a46be5-bc1c-490b-88ba-df9815e58c9b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.960952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51370ea6-9dd8-4214-bb8e-5609a4b40b8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.961596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51370ea6-9dd8-4214-bb8e-5609a4b40b8d service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.961959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51370ea6-9dd8-4214-bb8e-5609a4b40b8d service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.962342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51370ea6-9dd8-4214-bb8e-5609a4b40b8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.969594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2f8a62fd-7579-4a34-b2cb-7fba053e556a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:34.970820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2f8a62fd-7579-4a34-b2cb-7fba053e556a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:34.978954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 814/1615] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:34.980333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71735b4e-64c2-4e0c-9c9c-e4c06d3b93af None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.980620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71735b4e-64c2-4e0c-9c9c-e4c06d3b93af None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.980891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71735b4e-64c2-4e0c-9c9c-e4c06d3b93af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.018869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-71735b4e-64c2-4e0c-9c9c-e4c06d3b93af None None] MFA Rules not processed for user `31cb3c3b3b0a451980b09eecbb708a65`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:35.020415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-71735b4e-64c2-4e0c-9c9c-e4c06d3b93af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:35.038328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51370ea6-9dd8-4214-bb8e-5609a4b40b8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:35.040185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51370ea6-9dd8-4214-bb8e-5609a4b40b8d service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:35.052320 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 802/1616] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:35.072450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f7b1a-6a80-46a4-afe6-d7f138dbb871 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.072765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f7b1a-6a80-46a4-afe6-d7f138dbb871 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.076139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f7b1a-6a80-46a4-afe6-d7f138dbb871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.114021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 815/1617] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:35.115457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71735b4e-64c2-4e0c-9c9c-e4c06d3b93af None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:35.115920 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a56f7b1a-6a80-46a4-afe6-d7f138dbb871 None None] MFA Rules not processed for user `c7dd8b82a5284a9ea576a6ae27bc5664`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:35.117832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a56f7b1a-6a80-46a4-afe6-d7f138dbb871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:35.129248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71735b4e-64c2-4e0c-9c9c-e4c06d3b93af None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:35.134783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbab537-afb7-412c-8be3-0ec6abbebe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:35.135738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbab537-afb7-412c-8be3-0ec6abbebe60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.136223 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbab537-afb7-412c-8be3-0ec6abbebe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.136695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbab537-afb7-412c-8be3-0ec6abbebe60 admin admin] PATH_INFO: `/v3/projects/4f32b60f5fa8427386bafa1a91f4e608/users/962284e9a353430881bc119e5f80a890/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.154570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbab537-afb7-412c-8be3-0ec6abbebe60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f32b60f5fa8427386bafa1a91f4e608, user_id=962284e9a353430881bc119e5f80a890, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:35.157795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbab537-afb7-412c-8be3-0ec6abbebe60 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:35.187095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 816/1618] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:35 2022] PUT /identity/v3/projects/4f32b60f5fa8427386bafa1a91f4e608/users/962284e9a353430881bc119e5f80a890/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:35.189051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbab537-afb7-412c-8be3-0ec6abbebe60 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:35.257080 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:35.275090 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 803/1619] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:35.277180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f7b1a-6a80-46a4-afe6-d7f138dbb871 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:35.294345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbab537-afb7-412c-8be3-0ec6abbebe60 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:35.300241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f7b1a-6a80-46a4-afe6-d7f138dbb871 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:35.308932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbedc6c-57a9-4c53-bcd4-c25a6011e8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:35.311495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbedc6c-57a9-4c53-bcd4-c25a6011e8e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.313072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbedc6c-57a9-4c53-bcd4-c25a6011e8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.314681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbedc6c-57a9-4c53-bcd4-c25a6011e8e1 admin admin] PATH_INFO: `/v3/projects/79407ad98d864f7fa6c590cd9e0b093c/users/8545f30cccfd4e9b83b48d22fd7b34e5/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.333205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:35.374455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbedc6c-57a9-4c53-bcd4-c25a6011e8e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79407ad98d864f7fa6c590cd9e0b093c, user_id=8545f30cccfd4e9b83b48d22fd7b34e5, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:35.380779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbedc6c-57a9-4c53-bcd4-c25a6011e8e1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:35.411627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 817/1620] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:35 2022] PUT /identity/v3/projects/79407ad98d864f7fa6c590cd9e0b093c/users/8545f30cccfd4e9b83b48d22fd7b34e5/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:35.415185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbedc6c-57a9-4c53-bcd4-c25a6011e8e1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:35.417279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d2df6-a829-42ed-bfe9-5bf69f670ff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:35.418047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d2df6-a829-42ed-bfe9-5bf69f670ff1 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.418358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d2df6-a829-42ed-bfe9-5bf69f670ff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.418625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d2df6-a829-42ed-bfe9-5bf69f670ff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.443945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d2df6-a829-42ed-bfe9-5bf69f670ff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:35.445944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d2df6-a829-42ed-bfe9-5bf69f670ff1 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:35.459081 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 804/1621] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:35.460377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d209469-795f-4210-956b-5443c192ea54 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.460664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d209469-795f-4210-956b-5443c192ea54 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.460941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d209469-795f-4210-956b-5443c192ea54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.483945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3d209469-795f-4210-956b-5443c192ea54 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:35.485930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3d209469-795f-4210-956b-5443c192ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:35.554525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbedc6c-57a9-4c53-bcd4-c25a6011e8e1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:35.570610 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c22094-b54c-4d2b-98af-d40b6b719f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:35.571274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c22094-b54c-4d2b-98af-d40b6b719f14 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.571581 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c22094-b54c-4d2b-98af-d40b6b719f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.571849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c22094-b54c-4d2b-98af-d40b6b719f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.572489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c22094-b54c-4d2b-98af-d40b6b719f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:35.575178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c22094-b54c-4d2b-98af-d40b6b719f14 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:35.587789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 805/1622] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:35.590028 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca367f8-5ab8-4bf5-98ca-6fc930c9af4a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.599346 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca367f8-5ab8-4bf5-98ca-6fc930c9af4a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.599984 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca367f8-5ab8-4bf5-98ca-6fc930c9af4a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.604424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 806/1623] 10.222.0.86 () {54 vars in 962 bytes} [Mon Aug 29 19:01:35 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:35.606272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427c85f-f24b-49e4-acd9-1bdaf2a33c4c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.606571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427c85f-f24b-49e4-acd9-1bdaf2a33c4c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.606852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427c85f-f24b-49e4-acd9-1bdaf2a33c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.615023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 818/1624] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:35 2022] POST /identity/v3/projects => generated 363 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:35.616676 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2533b7-d87c-4afb-ac78-7141ffa3cd35 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.617225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2533b7-d87c-4afb-ac78-7141ffa3cd35 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.617688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2533b7-d87c-4afb-ac78-7141ffa3cd35 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.620633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 819/1625] 10.222.0.86 () {54 vars in 912 bytes} [Mon Aug 29 19:01:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:35.622173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2533b7-d87c-4afb-ac78-7141ffa3cd35 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:35.641884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2427c85f-f24b-49e4-acd9-1bdaf2a33c4c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:35.643163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2427c85f-f24b-49e4-acd9-1bdaf2a33c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:35.698220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2533b7-d87c-4afb-ac78-7141ffa3cd35 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:35.703800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf23401-6e8d-4de3-8403-f5062b23914c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:35.704419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf23401-6e8d-4de3-8403-f5062b23914c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.705094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf23401-6e8d-4de3-8403-f5062b23914c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.705094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf23401-6e8d-4de3-8403-f5062b23914c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.720759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 807/1626] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:35.727351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3972e7-6ed0-4409-bdf2-cf111f33fe91 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.727951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3972e7-6ed0-4409-bdf2-cf111f33fe91 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.728626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3972e7-6ed0-4409-bdf2-cf111f33fe91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.756153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ae3972e7-6ed0-4409-bdf2-cf111f33fe91 None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:35.757295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ae3972e7-6ed0-4409-bdf2-cf111f33fe91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:35.786011 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf23401-6e8d-4de3-8403-f5062b23914c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:35.788332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf23401-6e8d-4de3-8403-f5062b23914c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:35.795658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:35.806634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 820/1627] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:35.818349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf23401-6e8d-4de3-8403-f5062b23914c service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:35.838539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:35.846137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf23401-6e8d-4de3-8403-f5062b23914c service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:35.857791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216502c-5a24-4d51-ac38-4d64d5b962b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:35.858418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216502c-5a24-4d51-ac38-4d64d5b962b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.859871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216502c-5a24-4d51-ac38-4d64d5b962b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.860707 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216502c-5a24-4d51-ac38-4d64d5b962b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.861633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216502c-5a24-4d51-ac38-4d64d5b962b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:35.865594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216502c-5a24-4d51-ac38-4d64d5b962b8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:35.883154 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 808/1628] 10.222.0.86 () {58 vars in 1059 bytes} [Mon Aug 29 19:01:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:35.885831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5388a-e7a7-4128-8eba-4bd6928f882c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.886380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5388a-e7a7-4128-8eba-4bd6928f882c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.886672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5388a-e7a7-4128-8eba-4bd6928f882c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.922776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 821/1629] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:35 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:35.924035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216502c-5a24-4d51-ac38-4d64d5b962b8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:35.938910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-87d5388a-e7a7-4128-8eba-4bd6928f882c None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:35.940174 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-87d5388a-e7a7-4128-8eba-4bd6928f882c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:35.958417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216502c-5a24-4d51-ac38-4d64d5b962b8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:35.963468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f4f6b-df5b-41b4-b485-a5a3df926be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:35.964576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4f6b-df5b-41b4-b485-a5a3df926be5 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.965164 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4f6b-df5b-41b4-b485-a5a3df926be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.965745 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4f6b-df5b-41b4-b485-a5a3df926be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.975176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 809/1630] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:01:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:35.979023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5388a-e7a7-4128-8eba-4bd6928f882c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:35.996234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f4f6b-df5b-41b4-b485-a5a3df926be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:35.998642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f4f6b-df5b-41b4-b485-a5a3df926be5 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:36.010867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5388a-e7a7-4128-8eba-4bd6928f882c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:36.015900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 822/1631] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:36.019882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f4f6b-df5b-41b4-b485-a5a3df926be5 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:36.026256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c64e73-0b5a-408a-8c4e-5ebc41ef2225 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:36.026891 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c64e73-0b5a-408a-8c4e-5ebc41ef2225 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:36.028307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c64e73-0b5a-408a-8c4e-5ebc41ef2225 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:36.028307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c64e73-0b5a-408a-8c4e-5ebc41ef2225 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:36.040523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f4f6b-df5b-41b4-b485-a5a3df926be5 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:36.046885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae677e-e130-426a-883d-c12d3eed8769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:36.047972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae677e-e130-426a-883d-c12d3eed8769 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:36.048557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae677e-e130-426a-883d-c12d3eed8769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:36.049029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae677e-e130-426a-883d-c12d3eed8769 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:36.050364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae677e-e130-426a-883d-c12d3eed8769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:36.052384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae677e-e130-426a-883d-c12d3eed8769 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:36.070852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 823/1632] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:36.074421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae677e-e130-426a-883d-c12d3eed8769 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:36.134057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c64e73-0b5a-408a-8c4e-5ebc41ef2225 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:36.136922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c64e73-0b5a-408a-8c4e-5ebc41ef2225 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:36.150035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 810/1633] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:36.151035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c64e73-0b5a-408a-8c4e-5ebc41ef2225 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:36.163831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c64e73-0b5a-408a-8c4e-5ebc41ef2225 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:36.170395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375289a-af23-40c2-b9f1-1c0f6d4363a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:36.171295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375289a-af23-40c2-b9f1-1c0f6d4363a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:36.171648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375289a-af23-40c2-b9f1-1c0f6d4363a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:36.171996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375289a-af23-40c2-b9f1-1c0f6d4363a5 admin admin] PATH_INFO: `/v3/projects/140c9c3127124791acd2c9267aac62a5/users/5a5ffd70564f40beafd60758617d3cc9/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:36.193833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375289a-af23-40c2-b9f1-1c0f6d4363a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=140c9c3127124791acd2c9267aac62a5, user_id=5a5ffd70564f40beafd60758617d3cc9, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:36.197034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375289a-af23-40c2-b9f1-1c0f6d4363a5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:36.210928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae677e-e130-426a-883d-c12d3eed8769 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:36.217832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 811/1634] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:36 2022] PUT /identity/v3/projects/140c9c3127124791acd2c9267aac62a5/users/5a5ffd70564f40beafd60758617d3cc9/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:36.222256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375289a-af23-40c2-b9f1-1c0f6d4363a5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:36.344055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf255d-4742-4288-9072-2c0a1acd0771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:36.345047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf255d-4742-4288-9072-2c0a1acd0771 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:36.345577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf255d-4742-4288-9072-2c0a1acd0771 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:36.345879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf255d-4742-4288-9072-2c0a1acd0771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:36.354513 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375289a-af23-40c2-b9f1-1c0f6d4363a5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:36.367206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59a6ad-478d-4cc2-8860-18231857bb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:36.367842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59a6ad-478d-4cc2-8860-18231857bb25 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:36.370142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59a6ad-478d-4cc2-8860-18231857bb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:36.371503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59a6ad-478d-4cc2-8860-18231857bb25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:36.373368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59a6ad-478d-4cc2-8860-18231857bb25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:36.390270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59a6ad-478d-4cc2-8860-18231857bb25 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:36.420526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 812/1635] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:36.425319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59a6ad-478d-4cc2-8860-18231857bb25 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:36.450180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59a6ad-478d-4cc2-8860-18231857bb25 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:36.467904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d49e3d-1ca1-478b-88fe-9822f6d93c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:36.468598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d49e3d-1ca1-478b-88fe-9822f6d93c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:36.470601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d49e3d-1ca1-478b-88fe-9822f6d93c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:36.470905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d49e3d-1ca1-478b-88fe-9822f6d93c35 admin admin] PATH_INFO: `/v3/projects/140c9c3127124791acd2c9267aac62a5/users/5a5ffd70564f40beafd60758617d3cc9/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:36.477266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d49e3d-1ca1-478b-88fe-9822f6d93c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=140c9c3127124791acd2c9267aac62a5, user_id=5a5ffd70564f40beafd60758617d3cc9, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:36.484242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d49e3d-1ca1-478b-88fe-9822f6d93c35 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:36.515556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 813/1636] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:36 2022] PUT /identity/v3/projects/140c9c3127124791acd2c9267aac62a5/users/5a5ffd70564f40beafd60758617d3cc9/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:36.525824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f947afc-5de4-4cfb-bd86-8baf47f1cc80 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:36.526116 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f947afc-5de4-4cfb-bd86-8baf47f1cc80 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:36.526398 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f947afc-5de4-4cfb-bd86-8baf47f1cc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:36.538849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf255d-4742-4288-9072-2c0a1acd0771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:36.541836 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf255d-4742-4288-9072-2c0a1acd0771 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:36.574933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 824/1637] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:36.578704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c2684-719c-4ca7-8eec-25c2d80ddf75 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:36.579828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c2684-719c-4ca7-8eec-25c2d80ddf75 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:36.580114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c2684-719c-4ca7-8eec-25c2d80ddf75 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:36.587460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 825/1638] 10.222.0.86 () {54 vars in 912 bytes} [Mon Aug 29 19:01:36 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:36.591933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1f947afc-5de4-4cfb-bd86-8baf47f1cc80 None None] MFA Rules not processed for user `5a5ffd70564f40beafd60758617d3cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:36.593280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1f947afc-5de4-4cfb-bd86-8baf47f1cc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:36.607077 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99b8c2-02b0-4db8-8334-ff461a4f630d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:36.607324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99b8c2-02b0-4db8-8334-ff461a4f630d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:36.607414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99b8c2-02b0-4db8-8334-ff461a4f630d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:36.635240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:36.648198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8c99b8c2-02b0-4db8-8334-ff461a4f630d None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:36.650483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8c99b8c2-02b0-4db8-8334-ff461a4f630d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:36.673072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:36.776791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 814/1639] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:01:36.788901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f947afc-5de4-4cfb-bd86-8baf47f1cc80 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:36.803962 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 826/1640] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:01:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:36.927297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f947afc-5de4-4cfb-bd86-8baf47f1cc80 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:36.934957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17957a-c626-4704-8742-1f9b6da02775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:36.937168 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17957a-c626-4704-8742-1f9b6da02775 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:36.937433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17957a-c626-4704-8742-1f9b6da02775 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:36.937714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17957a-c626-4704-8742-1f9b6da02775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:36.962302 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17957a-c626-4704-8742-1f9b6da02775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:36.967217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17957a-c626-4704-8742-1f9b6da02775 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:36.998565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 815/1641] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:37.091193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99b8c2-02b0-4db8-8334-ff461a4f630d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:37.109477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99b8c2-02b0-4db8-8334-ff461a4f630d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:37.119601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb600a-29b3-46bb-bb95-e95373e672ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:37.120538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb600a-29b3-46bb-bb95-e95373e672ae service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:37.120977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb600a-29b3-46bb-bb95-e95373e672ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:37.121395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb600a-29b3-46bb-bb95-e95373e672ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:37.150633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb600a-29b3-46bb-bb95-e95373e672ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:37.152332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb600a-29b3-46bb-bb95-e95373e672ae service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:37.177771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 827/1642] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:37.689787 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17957a-c626-4704-8742-1f9b6da02775 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:37.701505 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17957a-c626-4704-8742-1f9b6da02775 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:37.708977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed91f48-08f7-4afb-9a7a-f9c6a46b501e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:37.710365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed91f48-08f7-4afb-9a7a-f9c6a46b501e service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:37.710704 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed91f48-08f7-4afb-9a7a-f9c6a46b501e service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:37.711710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed91f48-08f7-4afb-9a7a-f9c6a46b501e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:37.878520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed91f48-08f7-4afb-9a7a-f9c6a46b501e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:37.879992 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed91f48-08f7-4afb-9a7a-f9c6a46b501e service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:37.904660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 816/1643] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:38.841658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094f925-67db-4b08-8964-26e6a9eba996 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:38.842213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094f925-67db-4b08-8964-26e6a9eba996 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:38.842213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094f925-67db-4b08-8964-26e6a9eba996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:38.882443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d094f925-67db-4b08-8964-26e6a9eba996 None None] MFA Rules not processed for user `8545f30cccfd4e9b83b48d22fd7b34e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:38.883882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d094f925-67db-4b08-8964-26e6a9eba996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:39.010310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 828/1644] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:38 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:39.011714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed91f48-08f7-4afb-9a7a-f9c6a46b501e service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:39.109286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed91f48-08f7-4afb-9a7a-f9c6a46b501e service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:39.120883 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7ba7-c130-43de-9c7c-41705888fd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:39.121608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7ba7-c130-43de-9c7c-41705888fd44 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:39.123422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7ba7-c130-43de-9c7c-41705888fd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:39.123422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7ba7-c130-43de-9c7c-41705888fd44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:39.124636 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c7ba7-c130-43de-9c7c-41705888fd44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:39.127500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c7ba7-c130-43de-9c7c-41705888fd44 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:39.141754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:39.167044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 817/1645] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:39 2022] POST /identity/v3/projects => generated 405 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:39.170704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094f925-67db-4b08-8964-26e6a9eba996 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:39.194164 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094f925-67db-4b08-8964-26e6a9eba996 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:39.344220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b78c65-50da-4b4e-9a02-d06bdf1006d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:39.345172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b78c65-50da-4b4e-9a02-d06bdf1006d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:39.345679 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b78c65-50da-4b4e-9a02-d06bdf1006d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:39.346795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b78c65-50da-4b4e-9a02-d06bdf1006d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:39.347641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b78c65-50da-4b4e-9a02-d06bdf1006d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:39.357201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b78c65-50da-4b4e-9a02-d06bdf1006d2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:39.398080 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 829/1646] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:39 2022] POST /identity/v3/users => generated 309 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:39.401708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7ba7-c130-43de-9c7c-41705888fd44 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:39.421751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7ba7-c130-43de-9c7c-41705888fd44 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:39.445250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b2bca-fa9c-433a-b490-707be19ae5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:39.446157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b2bca-fa9c-433a-b490-707be19ae5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:39.446481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b2bca-fa9c-433a-b490-707be19ae5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:39.447169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b2bca-fa9c-433a-b490-707be19ae5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:39.447999 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b2bca-fa9c-433a-b490-707be19ae5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:39.453149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b2bca-fa9c-433a-b490-707be19ae5b7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:39.484072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 818/1647] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:39.488020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b78c65-50da-4b4e-9a02-d06bdf1006d2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:39.504790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b78c65-50da-4b4e-9a02-d06bdf1006d2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:39.514048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c2ffc-7632-4465-a6cd-ab50de1825b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:39.515786 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2ffc-7632-4465-a6cd-ab50de1825b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:39.516771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2ffc-7632-4465-a6cd-ab50de1825b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:39.517237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2ffc-7632-4465-a6cd-ab50de1825b4 admin admin] PATH_INFO: `/v3/projects/37a0cdcb10b94b579e728549f3275251/users/7616adb5f0df469e90a6b5c5309cf96c/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:39.574591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:39.574591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c2ffc-7632-4465-a6cd-ab50de1825b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a0cdcb10b94b579e728549f3275251, user_id=7616adb5f0df469e90a6b5c5309cf96c, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:39.574591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c2ffc-7632-4465-a6cd-ab50de1825b4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:39.598539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 830/1648] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:39 2022] PUT /identity/v3/projects/37a0cdcb10b94b579e728549f3275251/users/7616adb5f0df469e90a6b5c5309cf96c/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:39.599865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b2bca-fa9c-433a-b490-707be19ae5b7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:39.614903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b2bca-fa9c-433a-b490-707be19ae5b7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:39.798476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db7bf1-0564-4da9-9814-8f04fc0be891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:39.799480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db7bf1-0564-4da9-9814-8f04fc0be891 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:39.800196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db7bf1-0564-4da9-9814-8f04fc0be891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:39.800196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db7bf1-0564-4da9-9814-8f04fc0be891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:39.807655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db7bf1-0564-4da9-9814-8f04fc0be891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:39.813884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db7bf1-0564-4da9-9814-8f04fc0be891 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:39.840715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c2ffc-7632-4465-a6cd-ab50de1825b4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:39.843072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 819/1649] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:39.854133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c2ffc-7632-4465-a6cd-ab50de1825b4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:39.876157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d543df1-9d5e-4fbc-9e0b-c0873ceeea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:39.877035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d543df1-9d5e-4fbc-9e0b-c0873ceeea94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:39.877320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d543df1-9d5e-4fbc-9e0b-c0873ceeea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:39.877609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d543df1-9d5e-4fbc-9e0b-c0873ceeea94 admin admin] PATH_INFO: `/v3/projects/37a0cdcb10b94b579e728549f3275251/users/7616adb5f0df469e90a6b5c5309cf96c/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:39.883039 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d543df1-9d5e-4fbc-9e0b-c0873ceeea94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a0cdcb10b94b579e728549f3275251, user_id=7616adb5f0df469e90a6b5c5309cf96c, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:39.886583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d543df1-9d5e-4fbc-9e0b-c0873ceeea94 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:39.933858 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 831/1650] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:39 2022] PUT /identity/v3/projects/37a0cdcb10b94b579e728549f3275251/users/7616adb5f0df469e90a6b5c5309cf96c/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:43.837331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8443a2e-d03e-422d-84f3-8bdf5d843241 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:43.837818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8443a2e-d03e-422d-84f3-8bdf5d843241 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:43.837905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8443a2e-d03e-422d-84f3-8bdf5d843241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:43.917568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f8443a2e-d03e-422d-84f3-8bdf5d843241 None None] MFA Rules not processed for user `7616adb5f0df469e90a6b5c5309cf96c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:43.918345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f8443a2e-d03e-422d-84f3-8bdf5d843241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:44.080975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 820/1651] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:43 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:46.287960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa8a0a-9302-4b49-8efc-b5758f279a80 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:46.288521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa8a0a-9302-4b49-8efc-b5758f279a80 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:46.288521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa8a0a-9302-4b49-8efc-b5758f279a80 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:46.289794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 832/1652] 10.222.0.86 () {54 vars in 921 bytes} [Mon Aug 29 19:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:46.341882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be0231-601e-4d58-b037-82271641430a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:46.342799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be0231-601e-4d58-b037-82271641430a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:46.343353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be0231-601e-4d58-b037-82271641430a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:46.404314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-07be0231-601e-4d58-b037-82271641430a None None] MFA Rules not processed for user `27f771f20b734b679d5553d29a8a3884`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:46.405568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-07be0231-601e-4d58-b037-82271641430a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:46.542054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:46.587211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 821/1653] 10.222.0.86 () {58 vars in 1018 bytes} [Mon Aug 29 19:01:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:46.630940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa8a0a-9302-4b49-8efc-b5758f279a80 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:46.655954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa8a0a-9302-4b49-8efc-b5758f279a80 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:46.690505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e4f7e-8acd-4b69-b21a-0a663c9a45fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:46.691483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e4f7e-8acd-4b69-b21a-0a663c9a45fc service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:46.691483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e4f7e-8acd-4b69-b21a-0a663c9a45fc service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:46.691726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e4f7e-8acd-4b69-b21a-0a663c9a45fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:46.723197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e4f7e-8acd-4b69-b21a-0a663c9a45fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:46.724834 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e4f7e-8acd-4b69-b21a-0a663c9a45fc service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:46.766805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 833/1654] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:47.808782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d01eff-79db-4e79-a55f-c7d4b7deb997 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:47.809324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d01eff-79db-4e79-a55f-c7d4b7deb997 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:47.809959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d01eff-79db-4e79-a55f-c7d4b7deb997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:47.883565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-11d01eff-79db-4e79-a55f-c7d4b7deb997 None None] MFA Rules not processed for user `dd013ba83e944ecfa1a2d0a8a0d55101`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:47.885290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-11d01eff-79db-4e79-a55f-c7d4b7deb997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:47.974090 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 822/1655] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:47.989519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e4f7e-8acd-4b69-b21a-0a663c9a45fc service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:48.105848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e4f7e-8acd-4b69-b21a-0a663c9a45fc service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:48.113137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c763cc-4c2c-4361-95cc-5a973e801afb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:48.114025 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c763cc-4c2c-4361-95cc-5a973e801afb service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:48.114450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c763cc-4c2c-4361-95cc-5a973e801afb service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:48.115212 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c763cc-4c2c-4361-95cc-5a973e801afb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:48.126148 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c763cc-4c2c-4361-95cc-5a973e801afb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:48.128742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c763cc-4c2c-4361-95cc-5a973e801afb service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:48.140653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 834/1656] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:48.459453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180e0e4-9204-447f-a6d1-22c1a1c94830 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:48.459985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180e0e4-9204-447f-a6d1-22c1a1c94830 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:48.460420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180e0e4-9204-447f-a6d1-22c1a1c94830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:48.588694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9180e0e4-9204-447f-a6d1-22c1a1c94830 None None] MFA Rules not processed for user `5598718e51df4a7c92ef828de3003dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:48.593751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9180e0e4-9204-447f-a6d1-22c1a1c94830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:48.810386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 823/1657] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:48.813706 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47890fb-9f2c-4f0d-921e-a1407460e0ac None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:48.814129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47890fb-9f2c-4f0d-921e-a1407460e0ac None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:48.815357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47890fb-9f2c-4f0d-921e-a1407460e0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:48.841424 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a47890fb-9f2c-4f0d-921e-a1407460e0ac None None] MFA Rules not processed for user `5598718e51df4a7c92ef828de3003dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:48.843270 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a47890fb-9f2c-4f0d-921e-a1407460e0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:48.864859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 835/1658] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:48.870650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180e0e4-9204-447f-a6d1-22c1a1c94830 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:48.882800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0022677-a61b-4dfa-aed6-ca26a0d63ba2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:48.883333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0022677-a61b-4dfa-aed6-ca26a0d63ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:48.883746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0022677-a61b-4dfa-aed6-ca26a0d63ba2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:48.886650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 836/1659] 10.222.0.86 () {54 vars in 921 bytes} [Mon Aug 29 19:01:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:48.907716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180e0e4-9204-447f-a6d1-22c1a1c94830 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:48.915220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a05f7-fb1b-4d7d-9b52-725e9b5e0f45 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:48.915791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a05f7-fb1b-4d7d-9b52-725e9b5e0f45 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:48.916367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a05f7-fb1b-4d7d-9b52-725e9b5e0f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:48.940340 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f37ec9-2162-4b5e-b323-6a04983edf89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:48.940974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f37ec9-2162-4b5e-b323-6a04983edf89 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:48.941256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f37ec9-2162-4b5e-b323-6a04983edf89 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:48.941529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f37ec9-2162-4b5e-b323-6a04983edf89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:48.994112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f37ec9-2162-4b5e-b323-6a04983edf89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:49.000806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d95a05f7-fb1b-4d7d-9b52-725e9b5e0f45 None None] MFA Rules not processed for user `27f771f20b734b679d5553d29a8a3884`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:49.007423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d95a05f7-fb1b-4d7d-9b52-725e9b5e0f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:49.015516 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f37ec9-2162-4b5e-b323-6a04983edf89 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:49.040523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 837/1660] 10.222.0.86 () {58 vars in 1018 bytes} [Mon Aug 29 19:01:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:49.041780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a05f7-fb1b-4d7d-9b52-725e9b5e0f45 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:49.061446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 824/1661] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:49.062355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f37ec9-2162-4b5e-b323-6a04983edf89 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:49.161933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a05f7-fb1b-4d7d-9b52-725e9b5e0f45 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:49.168327 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:49.169434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:49.169865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:49.170426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] PATH_INFO: `/v3/users/8545f30cccfd4e9b83b48d22fd7b34e5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:49.171893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8545f30cccfd4e9b83b48d22fd7b34e5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:49.174802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:49.241036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:49.257510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8545f30cccfd4e9b83b48d22fd7b34e5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:49.258622 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8545f30cccfd4e9b83b48d22fd7b34e5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:49.301008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8545f30cccfd4e9b83b48d22fd7b34e5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:49.326828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8545f30cccfd4e9b83b48d22fd7b34e5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:49.341951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8545f30cccfd4e9b83b48d22fd7b34e5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:49.359503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 838/1662] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:01:49 2022] DELETE /identity/v3/users/8545f30cccfd4e9b83b48d22fd7b34e5 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:49.360804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:49.362495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:49.415450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f37ec9-2162-4b5e-b323-6a04983edf89 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:49.461116 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:49.571220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a3bd5-d781-4429-a5aa-f0aea157d3a5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:49.585176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a27b28-f135-4f78-ad0a-9be59a8438d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:49.592804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a27b28-f135-4f78-ad0a-9be59a8438d1 service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:49.593270 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a27b28-f135-4f78-ad0a-9be59a8438d1 service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:49.593550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a27b28-f135-4f78-ad0a-9be59a8438d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:49.601138 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-15a27b28-f135-4f78-ad0a-9be59a8438d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:49.624912 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:49.627064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:49.628056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:49.629150 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] PATH_INFO: `/v3/users/c7dd8b82a5284a9ea576a6ae27bc5664` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:49.632386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7dd8b82a5284a9ea576a6ae27bc5664)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:49.635433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:01:49.635433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Aug 29 19:01:49.635433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: Aug 29 19:01:49.637159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:49.645135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a27b28-f135-4f78-ad0a-9be59a8438d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:49.650335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a27b28-f135-4f78-ad0a-9be59a8438d1 service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:49.682855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 839/1663] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:49.818158 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7dd8b82a5284a9ea576a6ae27bc5664'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:49.850662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7dd8b82a5284a9ea576a6ae27bc5664'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:49.851946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7dd8b82a5284a9ea576a6ae27bc5664'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:49.852602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7dd8b82a5284a9ea576a6ae27bc5664'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:49.868803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7dd8b82a5284a9ea576a6ae27bc5664'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:49.924393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 825/1664] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:01:49 2022] DELETE /identity/v3/users/c7dd8b82a5284a9ea576a6ae27bc5664 => generated 0 bytes in 862 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:49.931356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287091c0-0431-450e-baa5-e644c97c13e5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:49.931903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287091c0-0431-450e-baa5-e644c97c13e5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:49.932387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287091c0-0431-450e-baa5-e644c97c13e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:49.980999 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-287091c0-0431-450e-baa5-e644c97c13e5 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:49.983831 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-287091c0-0431-450e-baa5-e644c97c13e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:49.998023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:49.999684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:50.159035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 840/1665] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:50.159515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23e63e-4268-4c12-8ba1-6d2b11f58df2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:50.164397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:50.165221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:50.165551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:50.165837 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] PATH_INFO: `/v3/projects/79407ad98d864f7fa6c590cd9e0b093c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:50.171595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79407ad98d864f7fa6c590cd9e0b093c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:50.173971 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:50.175155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287091c0-0431-450e-baa5-e644c97c13e5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:50.176691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-287091c0-0431-450e-baa5-e644c97c13e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:50.235683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:50.262404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79407ad98d864f7fa6c590cd9e0b093c', 'request_id': 'req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:50.275460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79407ad98d864f7fa6c590cd9e0b093c', 'request_id': 'req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:50.283552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:50.285809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79407ad98d864f7fa6c590cd9e0b093c', 'request_id': 'req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:50.302010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287091c0-0431-450e-baa5-e644c97c13e5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:50.326533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] The token cache is being invalidate because project 79407ad98d864f7fa6c590cd9e0b093c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:01:50.326861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:50.327947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:50.335559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 826/1666] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:01:49 2022] DELETE /identity/v3/projects/79407ad98d864f7fa6c590cd9e0b093c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:01:50.336726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:50.343224 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:50.445110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a42a4a-9d51-4b14-ac65-e7e4300db083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:50.445750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a42a4a-9d51-4b14-ac65-e7e4300db083 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:50.446027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a42a4a-9d51-4b14-ac65-e7e4300db083 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:50.446300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a42a4a-9d51-4b14-ac65-e7e4300db083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:50.448992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-62a42a4a-9d51-4b14-ac65-e7e4300db083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:50.487569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f6bc6-0fac-48ae-8f51-9ffca59c1f78 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:50.501959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad864bc-4a5f-4a29-aaca-0f9606b26810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:50.502632 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad864bc-4a5f-4a29-aaca-0f9606b26810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:50.502975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad864bc-4a5f-4a29-aaca-0f9606b26810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:50.503288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad864bc-4a5f-4a29-aaca-0f9606b26810 admin admin] PATH_INFO: `/v3/users/7616adb5f0df469e90a6b5c5309cf96c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:50.504822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad864bc-4a5f-4a29-aaca-0f9606b26810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7616adb5f0df469e90a6b5c5309cf96c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:50.511575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad864bc-4a5f-4a29-aaca-0f9606b26810 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:50.598849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a42a4a-9d51-4b14-ac65-e7e4300db083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:50.600826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a42a4a-9d51-4b14-ac65-e7e4300db083 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:50.629602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 841/1667] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:50.643277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bad864bc-4a5f-4a29-aaca-0f9606b26810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7616adb5f0df469e90a6b5c5309cf96c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:50.694582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bad864bc-4a5f-4a29-aaca-0f9606b26810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7616adb5f0df469e90a6b5c5309cf96c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:50.697487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bad864bc-4a5f-4a29-aaca-0f9606b26810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7616adb5f0df469e90a6b5c5309cf96c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:50.716779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bad864bc-4a5f-4a29-aaca-0f9606b26810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7616adb5f0df469e90a6b5c5309cf96c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:50.754688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bad864bc-4a5f-4a29-aaca-0f9606b26810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7616adb5f0df469e90a6b5c5309cf96c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:50.823729 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 827/1668] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:01:50 2022] DELETE /identity/v3/users/7616adb5f0df469e90a6b5c5309cf96c => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:01:50.867401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7d606-4df9-4532-8400-e4682cb22b14 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:50.867401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7d606-4df9-4532-8400-e4682cb22b14 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:50.867401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7d606-4df9-4532-8400-e4682cb22b14 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:50.870804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 842/1669] 10.222.0.86 () {54 vars in 912 bytes} [Mon Aug 29 19:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:50.884337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d901fb0-f032-4bc9-ad7a-dc08458c5dab None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:50.885111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d901fb0-f032-4bc9-ad7a-dc08458c5dab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:50.885230 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d901fb0-f032-4bc9-ad7a-dc08458c5dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:50.961808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c4f28-4d5d-490d-b9d5-919927b66290 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:50.961808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c4f28-4d5d-490d-b9d5-919927b66290 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:50.961808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c4f28-4d5d-490d-b9d5-919927b66290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:51.004075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9d901fb0-f032-4bc9-ad7a-dc08458c5dab None None] MFA Rules not processed for user `5ec64944b34d4ef1bd666bb1b7e216da`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:51.006876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9d901fb0-f032-4bc9-ad7a-dc08458c5dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:51.042319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-597c4f28-4d5d-490d-b9d5-919927b66290 None None] MFA Rules not processed for user `b114570010d94eea955b7b3f1fa97c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:51.044656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-597c4f28-4d5d-490d-b9d5-919927b66290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:51.047756 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:51.135142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 828/1670] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:01:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:51.144614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d901fb0-f032-4bc9-ad7a-dc08458c5dab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:51.159932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9d901fb0-f032-4bc9-ad7a-dc08458c5dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:51.290957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 843/1671] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:51.297905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c4f28-4d5d-490d-b9d5-919927b66290 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:51.301520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-597c4f28-4d5d-490d-b9d5-919927b66290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:51.380282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d901fb0-f032-4bc9-ad7a-dc08458c5dab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:51.392917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43677e6-a360-474a-870b-2f186be8acd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:51.393580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43677e6-a360-474a-870b-2f186be8acd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:51.393871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43677e6-a360-474a-870b-2f186be8acd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:51.394148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43677e6-a360-474a-870b-2f186be8acd4 admin admin] PATH_INFO: `/v3/users/cfe7c4e4e56e44d0aa562be2195bfdec` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:51.401215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43677e6-a360-474a-870b-2f186be8acd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe7c4e4e56e44d0aa562be2195bfdec)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:51.404752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43677e6-a360-474a-870b-2f186be8acd4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:51.415576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:51.559688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c4f28-4d5d-490d-b9d5-919927b66290 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:51.580381 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a43677e6-a360-474a-870b-2f186be8acd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe7c4e4e56e44d0aa562be2195bfdec'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:51.590276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1165f7d-c6eb-4212-a831-ba191698e967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:51.595518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1165f7d-c6eb-4212-a831-ba191698e967 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:51.595802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1165f7d-c6eb-4212-a831-ba191698e967 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:51.596082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1165f7d-c6eb-4212-a831-ba191698e967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:51.630003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1165f7d-c6eb-4212-a831-ba191698e967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:51.631948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1165f7d-c6eb-4212-a831-ba191698e967 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:51.633242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a43677e6-a360-474a-870b-2f186be8acd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe7c4e4e56e44d0aa562be2195bfdec'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:51.635135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a43677e6-a360-474a-870b-2f186be8acd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe7c4e4e56e44d0aa562be2195bfdec'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:51.636569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a43677e6-a360-474a-870b-2f186be8acd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe7c4e4e56e44d0aa562be2195bfdec'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:51.662794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 844/1672] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:01:51.663936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1165f7d-c6eb-4212-a831-ba191698e967 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:51.669524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a1165f7d-c6eb-4212-a831-ba191698e967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:51.674586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a43677e6-a360-474a-870b-2f186be8acd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe7c4e4e56e44d0aa562be2195bfdec'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:51.699171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1165f7d-c6eb-4212-a831-ba191698e967 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:51.710943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ae2a9-d269-4b18-94e9-1306b81a3a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:51.710943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ae2a9-d269-4b18-94e9-1306b81a3a43 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:51.711385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ae2a9-d269-4b18-94e9-1306b81a3a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:51.711730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ae2a9-d269-4b18-94e9-1306b81a3a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:51.711939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 829/1673] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:01:51 2022] DELETE /identity/v3/users/cfe7c4e4e56e44d0aa562be2195bfdec => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:51.715840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f22ae2a9-d269-4b18-94e9-1306b81a3a43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:51.718395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c028ed-c02c-42a6-98d8-bb1fe1d53977 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:51.730232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c028ed-c02c-42a6-98d8-bb1fe1d53977 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:51.730232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c028ed-c02c-42a6-98d8-bb1fe1d53977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:51.752063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ae2a9-d269-4b18-94e9-1306b81a3a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:51.753639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ae2a9-d269-4b18-94e9-1306b81a3a43 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:51.765218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 845/1674] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:51.781134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-77c028ed-c02c-42a6-98d8-bb1fe1d53977 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:51.782796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-77c028ed-c02c-42a6-98d8-bb1fe1d53977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:51.894815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:51.922735 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ae2a9-d269-4b18-94e9-1306b81a3a43 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:51.924317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f22ae2a9-d269-4b18-94e9-1306b81a3a43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:51.926822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 830/1675] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:51.954301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ae2a9-d269-4b18-94e9-1306b81a3a43 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:51.960435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:51.961432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:51.961709 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:51.961977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] PATH_INFO: `/v3/projects/37a0cdcb10b94b579e728549f3275251` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:51.963593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a0cdcb10b94b579e728549f3275251)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:51.964612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:01:51.964612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Aug 29 19:01:51.964612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: Aug 29 19:01:51.965797 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:51.978626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c028ed-c02c-42a6-98d8-bb1fe1d53977 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:51.980531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-77c028ed-c02c-42a6-98d8-bb1fe1d53977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:52.130703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a0cdcb10b94b579e728549f3275251', 'request_id': 'req-3524d85c-841a-4461-9a0c-7183310a5af7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:52.138428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c028ed-c02c-42a6-98d8-bb1fe1d53977 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:52.173200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a0cdcb10b94b579e728549f3275251', 'request_id': 'req-3524d85c-841a-4461-9a0c-7183310a5af7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:52.196298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a0cdcb10b94b579e728549f3275251', 'request_id': 'req-3524d85c-841a-4461-9a0c-7183310a5af7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:52.223877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] The token cache is being invalidate because project 37a0cdcb10b94b579e728549f3275251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:01:52.224221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:52.225145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3524d85c-841a-4461-9a0c-7183310a5af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:52.229948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 846/1676] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:01:51 2022] DELETE /identity/v3/projects/37a0cdcb10b94b579e728549f3275251 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:52.231532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e37d4-e284-44c3-ac76-3b84d4deafa3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:52.231982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e37d4-e284-44c3-ac76-3b84d4deafa3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:52.232087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e37d4-e284-44c3-ac76-3b84d4deafa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:52.275128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a3737-4bed-4e5b-ab26-f922d761cf52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:52.275894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a3737-4bed-4e5b-ab26-f922d761cf52 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:52.276184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a3737-4bed-4e5b-ab26-f922d761cf52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:52.276461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a3737-4bed-4e5b-ab26-f922d761cf52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:52.277115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-671e37d4-e284-44c3-ac76-3b84d4deafa3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:52.278365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-671e37d4-e284-44c3-ac76-3b84d4deafa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:52.281179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c75a3737-4bed-4e5b-ab26-f922d761cf52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:52.410443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a3737-4bed-4e5b-ab26-f922d761cf52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:52.412963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a3737-4bed-4e5b-ab26-f922d761cf52 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:52.425279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 847/1677] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:01:52.425279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e37d4-e284-44c3-ac76-3b84d4deafa3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:52.433724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-671e37d4-e284-44c3-ac76-3b84d4deafa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:52.461237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 831/1678] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:01:52.463749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaeafd-40ab-4f11-bc82-72049e45a44f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:52.464116 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaeafd-40ab-4f11-bc82-72049e45a44f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:52.464540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaeafd-40ab-4f11-bc82-72049e45a44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:52.498343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e37d4-e284-44c3-ac76-3b84d4deafa3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:52.529280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:52.535368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:52.535672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:52.535957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] PATH_INFO: `/v3/projects/4c10f21807bb4d53a03d1b3e11f365fe` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:52.540252 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c10f21807bb4d53a03d1b3e11f365fe)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:52.548025 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:52.551910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1eeaeafd-40ab-4f11-bc82-72049e45a44f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:52.560902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1eeaeafd-40ab-4f11-bc82-72049e45a44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:52.589435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 832/1679] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:52.595028 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaeafd-40ab-4f11-bc82-72049e45a44f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:52.630756 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaeafd-40ab-4f11-bc82-72049e45a44f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:52.651907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824414b6-e335-440d-b6e8-b20db2d18c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:52.652747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824414b6-e335-440d-b6e8-b20db2d18c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:52.653049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824414b6-e335-440d-b6e8-b20db2d18c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:52.653327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824414b6-e335-440d-b6e8-b20db2d18c74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:52.654100 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824414b6-e335-440d-b6e8-b20db2d18c74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:01:52.654407 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824414b6-e335-440d-b6e8-b20db2d18c74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:52.662055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824414b6-e335-440d-b6e8-b20db2d18c74 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:52.697652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:52.705033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 833/1680] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:01:52.710620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824414b6-e335-440d-b6e8-b20db2d18c74 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:52.736931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c10f21807bb4d53a03d1b3e11f365fe', 'request_id': 'req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:52.747818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824414b6-e335-440d-b6e8-b20db2d18c74 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:52.760610 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c10f21807bb4d53a03d1b3e11f365fe', 'request_id': 'req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:52.769461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c10f21807bb4d53a03d1b3e11f365fe', 'request_id': 'req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:52.798545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] The token cache is being invalidate because project 4c10f21807bb4d53a03d1b3e11f365fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:01:52.800304 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:52.802283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bc327ade-c366-4d4a-96f5-2e99d1b0b54f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:52.805457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 848/1681] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:01:52 2022] DELETE /identity/v3/projects/4c10f21807bb4d53a03d1b3e11f365fe => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:52.811263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14fa94-4c14-4b4d-888c-4e3af40d3a2d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:52.811942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14fa94-4c14-4b4d-888c-4e3af40d3a2d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:52.811942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14fa94-4c14-4b4d-888c-4e3af40d3a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:52.832545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:52.862337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cb14fa94-4c14-4b4d-888c-4e3af40d3a2d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:52.865345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442502c9-281a-453d-bfb9-350ede615327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:52.866197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442502c9-281a-453d-bfb9-350ede615327 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:52.866630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442502c9-281a-453d-bfb9-350ede615327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:52.867118 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442502c9-281a-453d-bfb9-350ede615327 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:52.868089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442502c9-281a-453d-bfb9-350ede615327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:52.870413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442502c9-281a-453d-bfb9-350ede615327 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:52.871883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cb14fa94-4c14-4b4d-888c-4e3af40d3a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:52.898473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 849/1682] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:52.915921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc2c28-19b2-4fa7-89f2-77da55599259 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:52.916510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc2c28-19b2-4fa7-89f2-77da55599259 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:52.916973 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc2c28-19b2-4fa7-89f2-77da55599259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:52.939141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 834/1683] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:52 2022] POST /identity/v3/projects => generated 355 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:52.942988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442502c9-281a-453d-bfb9-350ede615327 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:52.952347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-442502c9-281a-453d-bfb9-350ede615327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:52.996728 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c7cc2c28-19b2-4fa7-89f2-77da55599259 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:53.003310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c7cc2c28-19b2-4fa7-89f2-77da55599259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:53.174498 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442502c9-281a-453d-bfb9-350ede615327 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:53.188998 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f78db-3544-4913-ba00-05f7cb104625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:53.190451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f78db-3544-4913-ba00-05f7cb104625 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:53.190743 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f78db-3544-4913-ba00-05f7cb104625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:53.191036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f78db-3544-4913-ba00-05f7cb104625 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:53.193240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f78db-3544-4913-ba00-05f7cb104625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:53.199331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f78db-3544-4913-ba00-05f7cb104625 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:53.224372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 850/1684] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:53.229870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc2c28-19b2-4fa7-89f2-77da55599259 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:53.242255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc2c28-19b2-4fa7-89f2-77da55599259 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:53.267337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884af65-4dbb-4f21-a8e6-903d6b31dc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:53.271058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884af65-4dbb-4f21-a8e6-903d6b31dc7f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:53.271355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884af65-4dbb-4f21-a8e6-903d6b31dc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:53.271634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884af65-4dbb-4f21-a8e6-903d6b31dc7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:53.272333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884af65-4dbb-4f21-a8e6-903d6b31dc7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:01:53.272644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884af65-4dbb-4f21-a8e6-903d6b31dc7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:53.274306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884af65-4dbb-4f21-a8e6-903d6b31dc7f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:53.291531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 835/1685] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:52 2022] POST /identity/v3/users => generated 278 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:53.293660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f78db-3544-4913-ba00-05f7cb104625 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:53.329330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 851/1686] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:01:53.336067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884af65-4dbb-4f21-a8e6-903d6b31dc7f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:53.362776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f78db-3544-4913-ba00-05f7cb104625 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:53.367473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884af65-4dbb-4f21-a8e6-903d6b31dc7f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:53.387760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628686ea-6281-4904-9877-4abad1557559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:53.388394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628686ea-6281-4904-9877-4abad1557559 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:53.388676 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628686ea-6281-4904-9877-4abad1557559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:53.388949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628686ea-6281-4904-9877-4abad1557559 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:53.389576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628686ea-6281-4904-9877-4abad1557559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:53.392196 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628686ea-6281-4904-9877-4abad1557559 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:53.393007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f458428-aafd-48f1-abc8-0f6975b3b860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:53.393641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f458428-aafd-48f1-abc8-0f6975b3b860 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:53.393920 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f458428-aafd-48f1-abc8-0f6975b3b860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:53.394191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f458428-aafd-48f1-abc8-0f6975b3b860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:53.403180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f458428-aafd-48f1-abc8-0f6975b3b860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:53.410449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f458428-aafd-48f1-abc8-0f6975b3b860 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:53.455737 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 836/1687] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:53.457713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f458428-aafd-48f1-abc8-0f6975b3b860 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:53.463046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4f458428-aafd-48f1-abc8-0f6975b3b860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:53.482045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 852/1688] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:53 2022] POST /identity/v3/projects => generated 369 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:53.482045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628686ea-6281-4904-9877-4abad1557559 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:53.638691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f458428-aafd-48f1-abc8-0f6975b3b860 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:53.663216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628686ea-6281-4904-9877-4abad1557559 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:53.685868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf61991-34bc-4a8d-86de-3a132aa430f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:53.686909 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf61991-34bc-4a8d-86de-3a132aa430f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:53.687417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf61991-34bc-4a8d-86de-3a132aa430f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:53.687933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf61991-34bc-4a8d-86de-3a132aa430f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:53.688825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf61991-34bc-4a8d-86de-3a132aa430f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:53.691455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf61991-34bc-4a8d-86de-3a132aa430f2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:53.716054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 853/1689] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:53.717891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf61991-34bc-4a8d-86de-3a132aa430f2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:53.749719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf61991-34bc-4a8d-86de-3a132aa430f2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:53.754039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:53.757120 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b06379-402b-41dc-bbe2-456c78a8a64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:53.758045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b06379-402b-41dc-bbe2-456c78a8a64b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:53.758522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b06379-402b-41dc-bbe2-456c78a8a64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:53.759034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b06379-402b-41dc-bbe2-456c78a8a64b admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:53.760140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b06379-402b-41dc-bbe2-456c78a8a64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:53.762390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b06379-402b-41dc-bbe2-456c78a8a64b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:53.793855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f899bd-92aa-437f-b3be-3ccb6a4ebb54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:53.794495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f899bd-92aa-437f-b3be-3ccb6a4ebb54 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:53.794825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f899bd-92aa-437f-b3be-3ccb6a4ebb54 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:53.798003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f899bd-92aa-437f-b3be-3ccb6a4ebb54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:53.799692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:53.802204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-33f899bd-92aa-437f-b3be-3ccb6a4ebb54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:53.829654 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 854/1690] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:53 2022] POST /identity/v3/users => generated 285 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:53.831678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b06379-402b-41dc-bbe2-456c78a8a64b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:53.834384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f899bd-92aa-437f-b3be-3ccb6a4ebb54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:53.838872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f899bd-92aa-437f-b3be-3ccb6a4ebb54 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:53.850768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b06379-402b-41dc-bbe2-456c78a8a64b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:53.869992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d3e1d-eb04-462f-a72c-6de756a3cea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:53.871288 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d3e1d-eb04-462f-a72c-6de756a3cea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:53.871648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d3e1d-eb04-462f-a72c-6de756a3cea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:53.872394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d3e1d-eb04-462f-a72c-6de756a3cea9 admin admin] PATH_INFO: `/v3/projects/ec2fbf1e6d1140de858b2e1a3f52b71e/users/25a6f28de3e54d7ca1cd44cd9704fb04/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:53.892173 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 837/1691] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:53.893434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f899bd-92aa-437f-b3be-3ccb6a4ebb54 service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:53.911976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d3e1d-eb04-462f-a72c-6de756a3cea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2fbf1e6d1140de858b2e1a3f52b71e, user_id=25a6f28de3e54d7ca1cd44cd9704fb04, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:53.914885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d3e1d-eb04-462f-a72c-6de756a3cea9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:53.938003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f899bd-92aa-437f-b3be-3ccb6a4ebb54 service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:53.956567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a042db-82b3-4f8a-97b6-4c9f4df45e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:53.957195 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 855/1692] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:53 2022] PUT /identity/v3/projects/ec2fbf1e6d1140de858b2e1a3f52b71e/users/25a6f28de3e54d7ca1cd44cd9704fb04/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:53.957276 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a042db-82b3-4f8a-97b6-4c9f4df45e7e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:53.957644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a042db-82b3-4f8a-97b6-4c9f4df45e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:53.957826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a042db-82b3-4f8a-97b6-4c9f4df45e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:53.958363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674af87d-a60e-4282-bd33-bca2c8151e93 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:53.958464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a042db-82b3-4f8a-97b6-4c9f4df45e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:53.958647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674af87d-a60e-4282-bd33-bca2c8151e93 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:53.958926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674af87d-a60e-4282-bd33-bca2c8151e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:53.960133 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a042db-82b3-4f8a-97b6-4c9f4df45e7e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:53.985188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 838/1693] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:53.986072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34a69c-f5fa-49a4-ac5f-a90db3408f61 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:53.986748 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34a69c-f5fa-49a4-ac5f-a90db3408f61 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:53.987287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34a69c-f5fa-49a4-ac5f-a90db3408f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:54.059147 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-de34a69c-f5fa-49a4-ac5f-a90db3408f61 None None] MFA Rules not processed for user `25a6f28de3e54d7ca1cd44cd9704fb04`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:54.059937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-674af87d-a60e-4282-bd33-bca2c8151e93 None None] MFA Rules not processed for user `962284e9a353430881bc119e5f80a890`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:54.065307 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-674af87d-a60e-4282-bd33-bca2c8151e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:54.065624 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-de34a69c-f5fa-49a4-ac5f-a90db3408f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:54.257111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 839/1694] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:54.259268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34a69c-f5fa-49a4-ac5f-a90db3408f61 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:54.395674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 856/1695] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:54.412041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674af87d-a60e-4282-bd33-bca2c8151e93 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:54.453498 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34a69c-f5fa-49a4-ac5f-a90db3408f61 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:54.474200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d55df5-6d81-47a9-965e-22ea08a5e2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:54.474847 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d55df5-6d81-47a9-965e-22ea08a5e2bf admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:54.475161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d55df5-6d81-47a9-965e-22ea08a5e2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:54.475504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d55df5-6d81-47a9-965e-22ea08a5e2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:54.476116 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d55df5-6d81-47a9-965e-22ea08a5e2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:54.480731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d55df5-6d81-47a9-965e-22ea08a5e2bf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:54.521050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 840/1696] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:54.526723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164a411-44b3-4cdf-8065-6f69db2acc1c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:54.527046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164a411-44b3-4cdf-8065-6f69db2acc1c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:54.527337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164a411-44b3-4cdf-8065-6f69db2acc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:54.597540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b164a411-44b3-4cdf-8065-6f69db2acc1c None None] MFA Rules not processed for user `962284e9a353430881bc119e5f80a890`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:54.606244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b164a411-44b3-4cdf-8065-6f69db2acc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:54.618402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 841/1697] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:54.619495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164a411-44b3-4cdf-8065-6f69db2acc1c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:54.654473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164a411-44b3-4cdf-8065-6f69db2acc1c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:54.665816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674af87d-a60e-4282-bd33-bca2c8151e93 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:54.685479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7f6b3-0f6a-4151-bcd7-46ddacf9c7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:54.693162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7f6b3-0f6a-4151-bcd7-46ddacf9c7be admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:54.693497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7f6b3-0f6a-4151-bcd7-46ddacf9c7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:54.695056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7f6b3-0f6a-4151-bcd7-46ddacf9c7be admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:54.695894 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7f6b3-0f6a-4151-bcd7-46ddacf9c7be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:54.704098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7f6b3-0f6a-4151-bcd7-46ddacf9c7be admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:54.705979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40228312-4e02-44e9-ad82-6373afd7ad48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:54.708066 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40228312-4e02-44e9-ad82-6373afd7ad48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:54.709965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40228312-4e02-44e9-ad82-6373afd7ad48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:54.710254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40228312-4e02-44e9-ad82-6373afd7ad48 admin admin] PATH_INFO: `/v3/projects/e951777ddc97490f84efc416446b9945/users/c7d1c847e29f4272ab79bef320d32c9a/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:54.742637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40228312-4e02-44e9-ad82-6373afd7ad48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e951777ddc97490f84efc416446b9945, user_id=c7d1c847e29f4272ab79bef320d32c9a, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:54.748932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40228312-4e02-44e9-ad82-6373afd7ad48 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:54.799435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 842/1698] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:54 2022] PUT /identity/v3/projects/e951777ddc97490f84efc416446b9945/users/c7d1c847e29f4272ab79bef320d32c9a/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:54.800835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40228312-4e02-44e9-ad82-6373afd7ad48 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:54.803098 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-40228312-4e02-44e9-ad82-6373afd7ad48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:54.821057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 857/1699] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:54 2022] POST /identity/v3/projects => generated 355 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:54.821956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7f6b3-0f6a-4151-bcd7-46ddacf9c7be admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:54.825546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4bc7f6b3-0f6a-4151-bcd7-46ddacf9c7be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:54.867944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:54.950022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:54.978114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7f6b3-0f6a-4151-bcd7-46ddacf9c7be admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:54.984840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:54.985514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:54.985798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:54.986089 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] PATH_INFO: `/v3/projects/90b1cb2f10754e4d90caf3e3030c1eb4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:54.988531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90b1cb2f10754e4d90caf3e3030c1eb4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:54.990718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:55.006723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40228312-4e02-44e9-ad82-6373afd7ad48 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:55.127155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90b1cb2f10754e4d90caf3e3030c1eb4', 'request_id': 'req-81520895-0140-46cd-87c0-d829f40b24a4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:55.146917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90b1cb2f10754e4d90caf3e3030c1eb4', 'request_id': 'req-81520895-0140-46cd-87c0-d829f40b24a4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:55.168280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90b1cb2f10754e4d90caf3e3030c1eb4', 'request_id': 'req-81520895-0140-46cd-87c0-d829f40b24a4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:55.183119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0c466-1d5e-43c4-8e26-ed2491eb5f7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:55.185655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0c466-1d5e-43c4-8e26-ed2491eb5f7d service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:55.186114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0c466-1d5e-43c4-8e26-ed2491eb5f7d service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:55.186658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0c466-1d5e-43c4-8e26-ed2491eb5f7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:55.195628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] The token cache is being invalidate because project 90b1cb2f10754e4d90caf3e3030c1eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:01:55.195852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:55.196730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-81520895-0140-46cd-87c0-d829f40b24a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:01:55.202095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 858/1700] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:01:54 2022] DELETE /identity/v3/projects/90b1cb2f10754e4d90caf3e3030c1eb4 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:55.202095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8701d5-6422-4181-b898-d276d49beb3e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:55.210617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8701d5-6422-4181-b898-d276d49beb3e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:55.210976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8701d5-6422-4181-b898-d276d49beb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:55.276025 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cd8701d5-6422-4181-b898-d276d49beb3e None None] MFA Rules not processed for user `c7d1c847e29f4272ab79bef320d32c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:55.277278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cd8701d5-6422-4181-b898-d276d49beb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:55.296650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0c466-1d5e-43c4-8e26-ed2491eb5f7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:55.299321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0c466-1d5e-43c4-8e26-ed2491eb5f7d service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:55.339418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 843/1701] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:55.340702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0c466-1d5e-43c4-8e26-ed2491eb5f7d service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:55.368470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0c466-1d5e-43c4-8e26-ed2491eb5f7d service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:55.435145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 859/1702] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:55.435145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d59ed-75ca-4af3-92cb-843dd3570efc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:55.435145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d59ed-75ca-4af3-92cb-843dd3570efc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:55.435145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d59ed-75ca-4af3-92cb-843dd3570efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:55.482745 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5c9d59ed-75ca-4af3-92cb-843dd3570efc None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:55.485925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5c9d59ed-75ca-4af3-92cb-843dd3570efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:55.544214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:55.558942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1219c-96b9-4d3e-9b0b-492eedccfed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:55.559854 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1219c-96b9-4d3e-9b0b-492eedccfed2 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:55.560480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1219c-96b9-4d3e-9b0b-492eedccfed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:55.560967 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1219c-96b9-4d3e-9b0b-492eedccfed2 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:55.561809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1219c-96b9-4d3e-9b0b-492eedccfed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:55.568115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1219c-96b9-4d3e-9b0b-492eedccfed2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:55.615681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 860/1703] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:55.617099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4549a26-c505-4c0d-b060-01d30874647f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:55.617390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4549a26-c505-4c0d-b060-01d30874647f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:55.617664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4549a26-c505-4c0d-b060-01d30874647f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:55.632905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 844/1704] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:55 2022] POST /identity/v3/users => generated 284 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:55.634413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1219c-96b9-4d3e-9b0b-492eedccfed2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:55.645014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3ff1219c-96b9-4d3e-9b0b-492eedccfed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:55.649014 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:55.657211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b4549a26-c505-4c0d-b060-01d30874647f None None] MFA Rules not processed for user `dd013ba83e944ecfa1a2d0a8a0d55101`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:55.658311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b4549a26-c505-4c0d-b060-01d30874647f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:55.712551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1219c-96b9-4d3e-9b0b-492eedccfed2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:55.728099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd397ed-2708-41d9-92a6-848e9fb1ea13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:55.734522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd397ed-2708-41d9-92a6-848e9fb1ea13 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:55.734657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd397ed-2708-41d9-92a6-848e9fb1ea13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:55.735727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd397ed-2708-41d9-92a6-848e9fb1ea13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:55.737665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd397ed-2708-41d9-92a6-848e9fb1ea13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:55.742850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd397ed-2708-41d9-92a6-848e9fb1ea13 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:55.826176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 845/1705] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:55 2022] POST /identity/v3/projects => generated 369 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:55.826176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96349804-2bd8-4a66-951c-570eb706164e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:55.827254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 861/1706] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:55.827714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96349804-2bd8-4a66-951c-570eb706164e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:55.828084 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96349804-2bd8-4a66-951c-570eb706164e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:55.834074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4549a26-c505-4c0d-b060-01d30874647f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:55.838176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b4549a26-c505-4c0d-b060-01d30874647f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:55.884667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-96349804-2bd8-4a66-951c-570eb706164e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:55.885903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-96349804-2bd8-4a66-951c-570eb706164e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:55.888684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4549a26-c505-4c0d-b060-01d30874647f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.024183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee96af-a40a-459a-9401-d22daffb0700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.025596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee96af-a40a-459a-9401-d22daffb0700 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.025992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee96af-a40a-459a-9401-d22daffb0700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.026379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee96af-a40a-459a-9401-d22daffb0700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.027411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee96af-a40a-459a-9401-d22daffb0700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.036798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee96af-a40a-459a-9401-d22daffb0700 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.056892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 862/1707] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:56.058119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee96af-a40a-459a-9401-d22daffb0700 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.070637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 846/1708] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:56.074615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96349804-2bd8-4a66-951c-570eb706164e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.076082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee96af-a40a-459a-9401-d22daffb0700 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.086937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16935053-f651-4265-b1e9-724730946165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.088475 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16935053-f651-4265-b1e9-724730946165 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.088945 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16935053-f651-4265-b1e9-724730946165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.089422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16935053-f651-4265-b1e9-724730946165 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.090385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16935053-f651-4265-b1e9-724730946165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.096061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16935053-f651-4265-b1e9-724730946165 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.103722 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96349804-2bd8-4a66-951c-570eb706164e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.117063 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469f00b-2695-4444-bb07-fd0655eb1992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.118682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469f00b-2695-4444-bb07-fd0655eb1992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.119277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469f00b-2695-4444-bb07-fd0655eb1992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.119814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469f00b-2695-4444-bb07-fd0655eb1992 admin admin] PATH_INFO: `/v3/projects/723bc208147e439c88e9daec8f3b1a33/users/a24149ab7ddd4ec08ac9c0b3c0b9cefd/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.139859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 863/1709] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:56 2022] POST /identity/v3/users => generated 291 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:56.140830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16935053-f651-4265-b1e9-724730946165 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.157280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469f00b-2695-4444-bb07-fd0655eb1992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=723bc208147e439c88e9daec8f3b1a33, user_id=a24149ab7ddd4ec08ac9c0b3c0b9cefd, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.162206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469f00b-2695-4444-bb07-fd0655eb1992 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.168372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16935053-f651-4265-b1e9-724730946165 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.183500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23749d9-613f-4038-9814-46f4c78b63d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.184270 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23749d9-613f-4038-9814-46f4c78b63d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.184398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23749d9-613f-4038-9814-46f4c78b63d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.184589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23749d9-613f-4038-9814-46f4c78b63d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.184716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 847/1710] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:56 2022] PUT /identity/v3/projects/723bc208147e439c88e9daec8f3b1a33/users/a24149ab7ddd4ec08ac9c0b3c0b9cefd/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:56.185337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23749d9-613f-4038-9814-46f4c78b63d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:01:56.185602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469f00b-2695-4444-bb07-fd0655eb1992 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.185717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23749d9-613f-4038-9814-46f4c78b63d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.187307 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23749d9-613f-4038-9814-46f4c78b63d4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.214517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 864/1711] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:01:56.223495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23749d9-613f-4038-9814-46f4c78b63d4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.238711 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23749d9-613f-4038-9814-46f4c78b63d4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.335576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469f00b-2695-4444-bb07-fd0655eb1992 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.347984 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8af2f-6909-4c2c-baef-26cbd0f3b7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.348757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8af2f-6909-4c2c-baef-26cbd0f3b7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.349043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8af2f-6909-4c2c-baef-26cbd0f3b7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.349320 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8af2f-6909-4c2c-baef-26cbd0f3b7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.350151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a8af2f-6909-4c2c-baef-26cbd0f3b7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.352767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a8af2f-6909-4c2c-baef-26cbd0f3b7a0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.358903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f03150-905d-4772-90c6-a6246489ff55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.360983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f03150-905d-4772-90c6-a6246489ff55 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.361276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f03150-905d-4772-90c6-a6246489ff55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.361601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f03150-905d-4772-90c6-a6246489ff55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.362848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f03150-905d-4772-90c6-a6246489ff55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.365124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f03150-905d-4772-90c6-a6246489ff55 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.378065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 848/1712] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:56.378733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8af2f-6909-4c2c-baef-26cbd0f3b7a0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.386324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 865/1713] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:56.386324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f03150-905d-4772-90c6-a6246489ff55 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.404381 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8af2f-6909-4c2c-baef-26cbd0f3b7a0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.405818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f03150-905d-4772-90c6-a6246489ff55 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.412332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cff5a4-5c09-42fa-a5d6-ed0d8462e0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.413254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cff5a4-5c09-42fa-a5d6-ed0d8462e0a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.413689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cff5a4-5c09-42fa-a5d6-ed0d8462e0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.414135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cff5a4-5c09-42fa-a5d6-ed0d8462e0a8 admin admin] PATH_INFO: `/v3/projects/bf137add5a2245488c6a55cafa2ea143/users/dfa76489385f479abd77f0f18faec273/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.418980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f28b8-736d-4af1-9ee2-3f1f84f6455e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.419690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f28b8-736d-4af1-9ee2-3f1f84f6455e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.419905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f28b8-736d-4af1-9ee2-3f1f84f6455e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.420179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f28b8-736d-4af1-9ee2-3f1f84f6455e admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.421437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f28b8-736d-4af1-9ee2-3f1f84f6455e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.424065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f28b8-736d-4af1-9ee2-3f1f84f6455e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.427562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:56.432302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cff5a4-5c09-42fa-a5d6-ed0d8462e0a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf137add5a2245488c6a55cafa2ea143, user_id=dfa76489385f479abd77f0f18faec273, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.436488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cff5a4-5c09-42fa-a5d6-ed0d8462e0a8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.473003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 866/1714] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:56 2022] PUT /identity/v3/projects/bf137add5a2245488c6a55cafa2ea143/users/dfa76489385f479abd77f0f18faec273/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:56.475742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cff5a4-5c09-42fa-a5d6-ed0d8462e0a8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.499345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 849/1715] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:56 2022] POST /identity/v3/projects => generated 403 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:56.500811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f28b8-736d-4af1-9ee2-3f1f84f6455e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.536975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:56.614297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f28b8-736d-4af1-9ee2-3f1f84f6455e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.616450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cff5a4-5c09-42fa-a5d6-ed0d8462e0a8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.619737 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95015689-6698-4585-915b-9db0d0e9af4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.620355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95015689-6698-4585-915b-9db0d0e9af4f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.620637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95015689-6698-4585-915b-9db0d0e9af4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.620940 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95015689-6698-4585-915b-9db0d0e9af4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.621590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95015689-6698-4585-915b-9db0d0e9af4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.623221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95015689-6698-4585-915b-9db0d0e9af4f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.628190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51a784-c528-4c77-a791-9368550cd126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.629263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a784-c528-4c77-a791-9368550cd126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.630375 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a784-c528-4c77-a791-9368550cd126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.630784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a784-c528-4c77-a791-9368550cd126 admin admin] PATH_INFO: `/v3/projects/723bc208147e439c88e9daec8f3b1a33/users/a24149ab7ddd4ec08ac9c0b3c0b9cefd/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.634267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51a784-c528-4c77-a791-9368550cd126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=723bc208147e439c88e9daec8f3b1a33, user_id=a24149ab7ddd4ec08ac9c0b3c0b9cefd, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.638839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 850/1716] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:56.638839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95015689-6698-4585-915b-9db0d0e9af4f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.641699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51a784-c528-4c77-a791-9368550cd126 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.651266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95015689-6698-4585-915b-9db0d0e9af4f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.660766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed50a79-7f69-44f3-a523-1f905c6a1fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.661892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed50a79-7f69-44f3-a523-1f905c6a1fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.662383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed50a79-7f69-44f3-a523-1f905c6a1fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.662849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed50a79-7f69-44f3-a523-1f905c6a1fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.663698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed50a79-7f69-44f3-a523-1f905c6a1fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.665832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed50a79-7f69-44f3-a523-1f905c6a1fc0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.685468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 867/1717] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:56 2022] PUT /identity/v3/projects/723bc208147e439c88e9daec8f3b1a33/users/a24149ab7ddd4ec08ac9c0b3c0b9cefd/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:56.686491 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51a784-c528-4c77-a791-9368550cd126 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.708460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 851/1718] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:56 2022] POST /identity/v3/users => generated 302 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:56.709815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bca4c-bca1-4817-9c83-db0d9df0ffd2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.710111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bca4c-bca1-4817-9c83-db0d9df0ffd2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.710430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bca4c-bca1-4817-9c83-db0d9df0ffd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.776934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a67bca4c-bca1-4817-9c83-db0d9df0ffd2 None None] MFA Rules not processed for user `a24149ab7ddd4ec08ac9c0b3c0b9cefd`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:56.778194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a67bca4c-bca1-4817-9c83-db0d9df0ffd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:56.825139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51a784-c528-4c77-a791-9368550cd126 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:56.834158 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b1217-8938-42c3-a099-fcee5b129c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:56.835087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1217-8938-42c3-a099-fcee5b129c6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.835443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1217-8938-42c3-a099-fcee5b129c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.835782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1217-8938-42c3-a099-fcee5b129c6c admin admin] PATH_INFO: `/v3/projects/bf137add5a2245488c6a55cafa2ea143/users/dfa76489385f479abd77f0f18faec273/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.839258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b1217-8938-42c3-a099-fcee5b129c6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf137add5a2245488c6a55cafa2ea143, user_id=dfa76489385f479abd77f0f18faec273, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:56.846283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b1217-8938-42c3-a099-fcee5b129c6c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:56.873205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 868/1719] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:56 2022] PUT /identity/v3/projects/bf137add5a2245488c6a55cafa2ea143/users/dfa76489385f479abd77f0f18faec273/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:56.874317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b1217-8938-42c3-a099-fcee5b129c6c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:56.944150 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 852/1720] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:56 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:56.946319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a76b89-18ab-4999-b848-8a0ea2b208f3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.946616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a76b89-18ab-4999-b848-8a0ea2b208f3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.946898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a76b89-18ab-4999-b848-8a0ea2b208f3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:56.949163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 853/1721] 10.222.0.86 () {54 vars in 912 bytes} [Mon Aug 29 19:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:56.950396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ad62d-face-44d6-bbf7-30fd09356a44 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:56.950740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ad62d-face-44d6-bbf7-30fd09356a44 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:56.951042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ad62d-face-44d6-bbf7-30fd09356a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:57.001409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-041ad62d-face-44d6-bbf7-30fd09356a44 None None] MFA Rules not processed for user `dfa76489385f479abd77f0f18faec273`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:57.002751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-041ad62d-face-44d6-bbf7-30fd09356a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:57.040378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b1217-8938-42c3-a099-fcee5b129c6c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:57.054316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ef077-58d0-43de-8d53-be460753f558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:57.055053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef077-58d0-43de-8d53-be460753f558 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:57.055393 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef077-58d0-43de-8d53-be460753f558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:57.055669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef077-58d0-43de-8d53-be460753f558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:57.056313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ef077-58d0-43de-8d53-be460753f558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:57.058390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ef077-58d0-43de-8d53-be460753f558 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:57.085878 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 869/1722] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:57.085878 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ef077-58d0-43de-8d53-be460753f558 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:57.101484 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-874ef077-58d0-43de-8d53-be460753f558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:57.164908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:57.228416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ef077-58d0-43de-8d53-be460753f558 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:57.237743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c9e1b-5e7d-4cdb-b722-e66d0d2e3c01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:57.238572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c9e1b-5e7d-4cdb-b722-e66d0d2e3c01 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:57.238959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c9e1b-5e7d-4cdb-b722-e66d0d2e3c01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:57.239351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c9e1b-5e7d-4cdb-b722-e66d0d2e3c01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:57.281789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 854/1723] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:57.292019 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47635c7d-1a05-4b0f-affe-19297a1eb6c7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:57.292307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47635c7d-1a05-4b0f-affe-19297a1eb6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:57.292588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47635c7d-1a05-4b0f-affe-19297a1eb6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:57.326546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:57.354209 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-47635c7d-1a05-4b0f-affe-19297a1eb6c7 None None] MFA Rules not processed for user `5ec64944b34d4ef1bd666bb1b7e216da`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:57.365236 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-47635c7d-1a05-4b0f-affe-19297a1eb6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:57.432792 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c9e1b-5e7d-4cdb-b722-e66d0d2e3c01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:57.438742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c9e1b-5e7d-4cdb-b722-e66d0d2e3c01 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:57.464879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 870/1724] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:57.479050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c9e1b-5e7d-4cdb-b722-e66d0d2e3c01 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:57.527106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c9e1b-5e7d-4cdb-b722-e66d0d2e3c01 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:57.551589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea3d41-7aa0-4f99-872c-d4e983e6b695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:57.555058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea3d41-7aa0-4f99-872c-d4e983e6b695 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:57.558703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea3d41-7aa0-4f99-872c-d4e983e6b695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:57.559439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea3d41-7aa0-4f99-872c-d4e983e6b695 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:57.561389 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea3d41-7aa0-4f99-872c-d4e983e6b695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:57.573246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea3d41-7aa0-4f99-872c-d4e983e6b695 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:57.618112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 855/1725] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:01:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:57.632384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 871/1726] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:57.635517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea3d41-7aa0-4f99-872c-d4e983e6b695 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:57.646417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-23ea3d41-7aa0-4f99-872c-d4e983e6b695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:57.647628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da4a56-b2d1-4dd3-a10d-f6fa59857854 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:57.650609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da4a56-b2d1-4dd3-a10d-f6fa59857854 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:57.651816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da4a56-b2d1-4dd3-a10d-f6fa59857854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:57.692139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea3d41-7aa0-4f99-872c-d4e983e6b695 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:57.698034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396a1ed-5ccc-47e6-9b7a-f7fc8030fe7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:57.698845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396a1ed-5ccc-47e6-9b7a-f7fc8030fe7b service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:57.699244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396a1ed-5ccc-47e6-9b7a-f7fc8030fe7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:57.699447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396a1ed-5ccc-47e6-9b7a-f7fc8030fe7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:57.713414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396a1ed-5ccc-47e6-9b7a-f7fc8030fe7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:57.715538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396a1ed-5ccc-47e6-9b7a-f7fc8030fe7b service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:57.727955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 872/1727] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:57.729155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396a1ed-5ccc-47e6-9b7a-f7fc8030fe7b service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:57.731142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8396a1ed-5ccc-47e6-9b7a-f7fc8030fe7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:57.752641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-28da4a56-b2d1-4dd3-a10d-f6fa59857854 None None] MFA Rules not processed for user `b114570010d94eea955b7b3f1fa97c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:57.756254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-28da4a56-b2d1-4dd3-a10d-f6fa59857854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:57.760731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396a1ed-5ccc-47e6-9b7a-f7fc8030fe7b service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:57.769249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792cb7c-4686-4b0b-94c7-4d1355bddbd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:57.770273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792cb7c-4686-4b0b-94c7-4d1355bddbd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:57.770769 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792cb7c-4686-4b0b-94c7-4d1355bddbd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:57.771204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792cb7c-4686-4b0b-94c7-4d1355bddbd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:57.791904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792cb7c-4686-4b0b-94c7-4d1355bddbd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:57.802936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792cb7c-4686-4b0b-94c7-4d1355bddbd7 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:57.834953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 873/1728] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:57.842804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792cb7c-4686-4b0b-94c7-4d1355bddbd7 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:57.876253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792cb7c-4686-4b0b-94c7-4d1355bddbd7 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:57.900953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621ff5a-6547-4492-a803-a536bf53f7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:57.901852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621ff5a-6547-4492-a803-a536bf53f7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:57.902289 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621ff5a-6547-4492-a803-a536bf53f7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:57.902743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621ff5a-6547-4492-a803-a536bf53f7a8 admin admin] PATH_INFO: `/v3/projects/0b66b8cd296344679f5f027db4a847fa/users/90a1837d530a4034b3c4541221f66526/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:57.940282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621ff5a-6547-4492-a803-a536bf53f7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b66b8cd296344679f5f027db4a847fa, user_id=90a1837d530a4034b3c4541221f66526, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:57.946458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621ff5a-6547-4492-a803-a536bf53f7a8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:57.987591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 856/1729] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:57.987591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23394846-6fa0-4ed7-b3d7-637263572aa5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:57.987591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23394846-6fa0-4ed7-b3d7-637263572aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:57.987591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23394846-6fa0-4ed7-b3d7-637263572aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:58.003029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 874/1730] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:57 2022] PUT /identity/v3/projects/0b66b8cd296344679f5f027db4a847fa/users/90a1837d530a4034b3c4541221f66526/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:01:58.004111 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621ff5a-6547-4492-a803-a536bf53f7a8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:58.104553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-23394846-6fa0-4ed7-b3d7-637263572aa5 None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:58.108104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-23394846-6fa0-4ed7-b3d7-637263572aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:58.208142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621ff5a-6547-4492-a803-a536bf53f7a8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:58.222885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cb805-cdce-437c-85f7-ad838332bf32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:58.223954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cb805-cdce-437c-85f7-ad838332bf32 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:58.224333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cb805-cdce-437c-85f7-ad838332bf32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:58.224861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cb805-cdce-437c-85f7-ad838332bf32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:58.266452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:58.318468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 857/1731] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:01:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:58.319500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23394846-6fa0-4ed7-b3d7-637263572aa5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:58.337302 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23394846-6fa0-4ed7-b3d7-637263572aa5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:58.342223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2adaf-01bd-44e9-ba13-bc5dbd4ae4d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:58.343254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2adaf-01bd-44e9-ba13-bc5dbd4ae4d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:58.343378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2adaf-01bd-44e9-ba13-bc5dbd4ae4d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:58.344457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2adaf-01bd-44e9-ba13-bc5dbd4ae4d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:58.347492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d6a2adaf-01bd-44e9-ba13-bc5dbd4ae4d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:58.428067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:58.436799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6cb805-cdce-437c-85f7-ad838332bf32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:58.439746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6cb805-cdce-437c-85f7-ad838332bf32 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:58.501779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 875/1732] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:58.506449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff1923-b397-4b24-8368-dfd057bb37c1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:58.506545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2adaf-01bd-44e9-ba13-bc5dbd4ae4d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:58.507042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff1923-b397-4b24-8368-dfd057bb37c1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:58.507469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff1923-b397-4b24-8368-dfd057bb37c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:58.511992 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2adaf-01bd-44e9-ba13-bc5dbd4ae4d5 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:58.559087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 858/1733] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:58.571944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807f874-cf08-4ffa-a827-d9337b498225 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:58.572111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807f874-cf08-4ffa-a827-d9337b498225 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:58.572389 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807f874-cf08-4ffa-a827-d9337b498225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:58.611089 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d9ff1923-b397-4b24-8368-dfd057bb37c1 None None] MFA Rules not processed for user `90a1837d530a4034b3c4541221f66526`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:58.613560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d9ff1923-b397-4b24-8368-dfd057bb37c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:58.643635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0807f874-cf08-4ffa-a827-d9337b498225 None None] MFA Rules not processed for user `962284e9a353430881bc119e5f80a890`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:58.644894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0807f874-cf08-4ffa-a827-d9337b498225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:58.837743 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 859/1734] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:58.838922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807f874-cf08-4ffa-a827-d9337b498225 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:58.846806 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0807f874-cf08-4ffa-a827-d9337b498225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:58.989360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 876/1735] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:01:58 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:58.996031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff1923-b397-4b24-8368-dfd057bb37c1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:59.218038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff1923-b397-4b24-8368-dfd057bb37c1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:59.239301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135182a-f752-42ab-abf8-095c5ec67f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:59.241277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135182a-f752-42ab-abf8-095c5ec67f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:59.241511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135182a-f752-42ab-abf8-095c5ec67f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:59.241789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135182a-f752-42ab-abf8-095c5ec67f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:59.244965 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135182a-f752-42ab-abf8-095c5ec67f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:59.248441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135182a-f752-42ab-abf8-095c5ec67f30 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:59.274545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807f874-cf08-4ffa-a827-d9337b498225 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:59.305164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d0e3a-2012-47e6-a215-f281380efdea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:59.306325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d0e3a-2012-47e6-a215-f281380efdea service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:59.321501 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d0e3a-2012-47e6-a215-f281380efdea service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:59.322634 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d0e3a-2012-47e6-a215-f281380efdea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:59.336533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 877/1736] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:01:58 2022] POST /identity/v3/projects => generated 405 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:59.349389 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135182a-f752-42ab-abf8-095c5ec67f30 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:59.377671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d0e3a-2012-47e6-a215-f281380efdea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:59.379659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d0e3a-2012-47e6-a215-f281380efdea service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:59.423859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 860/1737] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:01:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:01:59.595807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:59.726966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135182a-f752-42ab-abf8-095c5ec67f30 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:59.747141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f84cf-9e36-408d-9d80-4c83bd196402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:59.747657 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f84cf-9e36-408d-9d80-4c83bd196402 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:59.748070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f84cf-9e36-408d-9d80-4c83bd196402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:59.748239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f84cf-9e36-408d-9d80-4c83bd196402 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:59.748856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f84cf-9e36-408d-9d80-4c83bd196402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:59.751866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f84cf-9e36-408d-9d80-4c83bd196402 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:59.769077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d0e3a-2012-47e6-a215-f281380efdea service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:59.770747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8b9d0e3a-2012-47e6-a215-f281380efdea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:59.801613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d0e3a-2012-47e6-a215-f281380efdea service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:59.823337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c097f5a-4b2e-48a6-b476-e60e1f22020a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:59.824426 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097f5a-4b2e-48a6-b476-e60e1f22020a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:59.825193 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097f5a-4b2e-48a6-b476-e60e1f22020a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:59.825193 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097f5a-4b2e-48a6-b476-e60e1f22020a admin admin] PATH_INFO: `/v3/users/31cb3c3b3b0a451980b09eecbb708a65` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:59.839128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c097f5a-4b2e-48a6-b476-e60e1f22020a admin admin] RBAC: Authorizing `identity:delete_user(user_id=31cb3c3b3b0a451980b09eecbb708a65)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:59.839128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c097f5a-4b2e-48a6-b476-e60e1f22020a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:59.839887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 878/1738] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:01:59 2022] POST /identity/v3/users => generated 309 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:01:59.841347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f84cf-9e36-408d-9d80-4c83bd196402 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:59.865632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f84cf-9e36-408d-9d80-4c83bd196402 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:59.878076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983741b2-1040-46be-b90d-14024ba79545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:59.880259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983741b2-1040-46be-b90d-14024ba79545 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:59.882273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983741b2-1040-46be-b90d-14024ba79545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:59.884043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983741b2-1040-46be-b90d-14024ba79545 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:59.887053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983741b2-1040-46be-b90d-14024ba79545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:59.891847 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983741b2-1040-46be-b90d-14024ba79545 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:59.928015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 879/1739] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:01:59.929879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983741b2-1040-46be-b90d-14024ba79545 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:59.954292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983741b2-1040-46be-b90d-14024ba79545 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:59.976133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140f32e-e024-4def-a5ae-bf6ec4a82a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:59.977687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140f32e-e024-4def-a5ae-bf6ec4a82a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:59.977687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140f32e-e024-4def-a5ae-bf6ec4a82a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:59.977687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140f32e-e024-4def-a5ae-bf6ec4a82a19 admin admin] PATH_INFO: `/v3/projects/8c3428a94007405ca776b211c15375c5/users/4dbced0836e649d99d358dc555da87e8/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:00.007736 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:00.035727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140f32e-e024-4def-a5ae-bf6ec4a82a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c3428a94007405ca776b211c15375c5, user_id=4dbced0836e649d99d358dc555da87e8, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:00.036892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0c097f5a-4b2e-48a6-b476-e60e1f22020a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31cb3c3b3b0a451980b09eecbb708a65'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:00.039204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140f32e-e024-4def-a5ae-bf6ec4a82a19 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:00.076978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0c097f5a-4b2e-48a6-b476-e60e1f22020a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31cb3c3b3b0a451980b09eecbb708a65'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:00.089909 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 880/1740] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:01:59 2022] PUT /identity/v3/projects/8c3428a94007405ca776b211c15375c5/users/4dbced0836e649d99d358dc555da87e8/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:00.090986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140f32e-e024-4def-a5ae-bf6ec4a82a19 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:00.101574 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e140f32e-e024-4def-a5ae-bf6ec4a82a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:00.101729 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0c097f5a-4b2e-48a6-b476-e60e1f22020a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31cb3c3b3b0a451980b09eecbb708a65'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:00.107774 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0c097f5a-4b2e-48a6-b476-e60e1f22020a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31cb3c3b3b0a451980b09eecbb708a65'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:00.152577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0c097f5a-4b2e-48a6-b476-e60e1f22020a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31cb3c3b3b0a451980b09eecbb708a65'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:00.237003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 861/1741] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:01:59 2022] DELETE /identity/v3/users/31cb3c3b3b0a451980b09eecbb708a65 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:00.238625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45611f0-ba59-4c76-a5be-657a3592c5ca None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:00.238926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45611f0-ba59-4c76-a5be-657a3592c5ca None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:00.244945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45611f0-ba59-4c76-a5be-657a3592c5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:00.317204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a45611f0-ba59-4c76-a5be-657a3592c5ca None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:00.322599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a45611f0-ba59-4c76-a5be-657a3592c5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:00.408977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140f32e-e024-4def-a5ae-bf6ec4a82a19 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:00.621733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 862/1742] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:00.693050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45611f0-ba59-4c76-a5be-657a3592c5ca None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:00.694710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a45611f0-ba59-4c76-a5be-657a3592c5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:00.845155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ac71a-8b04-4d0d-b005-30a936975d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:00.845859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ac71a-8b04-4d0d-b005-30a936975d28 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:00.846155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ac71a-8b04-4d0d-b005-30a936975d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:00.846416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ac71a-8b04-4d0d-b005-30a936975d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:00.847103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ac71a-8b04-4d0d-b005-30a936975d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:00.848727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ac71a-8b04-4d0d-b005-30a936975d28 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:00.911072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 881/1743] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 820 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:00.919376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c7829-45b8-4e55-be62-ec3fbf8492c2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:00.919376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c7829-45b8-4e55-be62-ec3fbf8492c2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:00.919376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c7829-45b8-4e55-be62-ec3fbf8492c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:00.963009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45611f0-ba59-4c76-a5be-657a3592c5ca None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:00.971574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e4ae4-c4eb-49e9-b5d1-078a230ec060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:00.972737 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e4ae4-c4eb-49e9-b5d1-078a230ec060 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:00.975128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e4ae4-c4eb-49e9-b5d1-078a230ec060 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:00.975128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e4ae4-c4eb-49e9-b5d1-078a230ec060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:00.990241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e4ae4-c4eb-49e9-b5d1-078a230ec060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:00.994421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e4ae4-c4eb-49e9-b5d1-078a230ec060 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:01.003752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-158c7829-45b8-4e55-be62-ec3fbf8492c2 None None] MFA Rules not processed for user `5ec64944b34d4ef1bd666bb1b7e216da`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:01.005007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-158c7829-45b8-4e55-be62-ec3fbf8492c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:01.014690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 863/1744] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:01.015992 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e4ae4-c4eb-49e9-b5d1-078a230ec060 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:01.028974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e4ae4-c4eb-49e9-b5d1-078a230ec060 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:01.036733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f6f10-859b-4132-a2d9-d5837a33bb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:01.037778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f6f10-859b-4132-a2d9-d5837a33bb2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:01.038303 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f6f10-859b-4132-a2d9-d5837a33bb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:01.038796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f6f10-859b-4132-a2d9-d5837a33bb2b admin admin] PATH_INFO: `/v3/projects/8c3428a94007405ca776b211c15375c5/users/4dbced0836e649d99d358dc555da87e8/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:01.041705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f6f10-859b-4132-a2d9-d5837a33bb2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c3428a94007405ca776b211c15375c5, user_id=4dbced0836e649d99d358dc555da87e8, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:01.045275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 882/1745] 10.222.0.89 () {58 vars in 1015 bytes} [Mon Aug 29 19:02:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:01.048476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f6f10-859b-4132-a2d9-d5837a33bb2b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:01.067039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 864/1746] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:01 2022] PUT /identity/v3/projects/8c3428a94007405ca776b211c15375c5/users/4dbced0836e649d99d358dc555da87e8/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:01.078070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1dad0-7a88-4616-8bc7-cf8fd9b484e9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:01.078350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1dad0-7a88-4616-8bc7-cf8fd9b484e9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:01.079753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1dad0-7a88-4616-8bc7-cf8fd9b484e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:01.175300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a7b1dad0-7a88-4616-8bc7-cf8fd9b484e9 None None] MFA Rules not processed for user `4dbced0836e649d99d358dc555da87e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:01.177173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a7b1dad0-7a88-4616-8bc7-cf8fd9b484e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:01.211086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:01.486635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 883/1747] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:01 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:01.497514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f6f10-859b-4132-a2d9-d5837a33bb2b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:01.499269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-354f6f10-859b-4132-a2d9-d5837a33bb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:01.515272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1dad0-7a88-4616-8bc7-cf8fd9b484e9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:01.717250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:01.747121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f6f10-859b-4132-a2d9-d5837a33bb2b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:01.759746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc452b-37e2-49ef-9c6c-7b1cf970ce7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:01.760381 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc452b-37e2-49ef-9c6c-7b1cf970ce7c service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:01.760666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc452b-37e2-49ef-9c6c-7b1cf970ce7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:01.760914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc452b-37e2-49ef-9c6c-7b1cf970ce7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:01.793152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1dad0-7a88-4616-8bc7-cf8fd9b484e9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:01.797443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc452b-37e2-49ef-9c6c-7b1cf970ce7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:01.798948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc452b-37e2-49ef-9c6c-7b1cf970ce7c service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:01.813247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e331455-3897-4083-81af-abe6108d7ee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:01.820852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e331455-3897-4083-81af-abe6108d7ee8 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:01.826056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e331455-3897-4083-81af-abe6108d7ee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:01.826519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e331455-3897-4083-81af-abe6108d7ee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:01.898097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 865/1748] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:01 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:01.900589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e331455-3897-4083-81af-abe6108d7ee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:01.910906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e331455-3897-4083-81af-abe6108d7ee8 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:01.942910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 884/1749] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:02.470664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc452b-37e2-49ef-9c6c-7b1cf970ce7c service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:02.746613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc452b-37e2-49ef-9c6c-7b1cf970ce7c service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:02.791477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:02.792485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:02.793291 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:02.793799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] PATH_INFO: `/v3/users/90a1837d530a4034b3c4541221f66526` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:02.796148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=90a1837d530a4034b3c4541221f66526)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:02.798437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:02.981248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90a1837d530a4034b3c4541221f66526'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:03.047571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90a1837d530a4034b3c4541221f66526'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:03.049638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90a1837d530a4034b3c4541221f66526'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:03.079069 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90a1837d530a4034b3c4541221f66526'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:03.126559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90a1837d530a4034b3c4541221f66526'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:03.255092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 866/1750] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:02 2022] DELETE /identity/v3/users/90a1837d530a4034b3c4541221f66526 => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:03.260391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b391-9846-4f10-978c-a6e39f1db7f9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:03.261510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b391-9846-4f10-978c-a6e39f1db7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:03.261994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b391-9846-4f10-978c-a6e39f1db7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:03.375410 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b254b391-9846-4f10-978c-a6e39f1db7f9 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:03.376950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b254b391-9846-4f10-978c-a6e39f1db7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:03.760512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 885/1751] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:03.821316 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:03.823833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:04.081700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b3c5e-7ce8-4183-81da-b0eb369760bd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:04.097721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d246c-c51b-4aed-bd22-99c58d068b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:04.098392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d246c-c51b-4aed-bd22-99c58d068b0d service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:04.098673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d246c-c51b-4aed-bd22-99c58d068b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:04.100474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d246c-c51b-4aed-bd22-99c58d068b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:04.144884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d246c-c51b-4aed-bd22-99c58d068b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:04.148250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d246c-c51b-4aed-bd22-99c58d068b0d service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:04.173477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254b391-9846-4f10-978c-a6e39f1db7f9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:04.175166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b254b391-9846-4f10-978c-a6e39f1db7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:04.223587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254b391-9846-4f10-978c-a6e39f1db7f9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:04.247713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 867/1752] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:04.259902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:04.260957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:04.261697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:04.266032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] PATH_INFO: `/v3/projects/3cb291f3145d45feb6e34d6674ce29a9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:04.267951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cb291f3145d45feb6e34d6674ce29a9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:04.276156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:04.610271 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:04.622086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cb291f3145d45feb6e34d6674ce29a9', 'request_id': 'req-42ae4af8-129f-41c4-9506-e7c5d28d56a3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:04.658477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cb291f3145d45feb6e34d6674ce29a9', 'request_id': 'req-42ae4af8-129f-41c4-9506-e7c5d28d56a3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:04.677207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cb291f3145d45feb6e34d6674ce29a9', 'request_id': 'req-42ae4af8-129f-41c4-9506-e7c5d28d56a3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:04.725142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] The token cache is being invalidate because project 3cb291f3145d45feb6e34d6674ce29a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:04.728562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:04.744250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:04.754710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 886/1753] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:04 2022] DELETE /identity/v3/projects/3cb291f3145d45feb6e34d6674ce29a9 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:04.755552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d246c-c51b-4aed-bd22-99c58d068b0d service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:04.772286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d246c-c51b-4aed-bd22-99c58d068b0d service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:04.818559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:05.158087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df69ba-4eb5-42a0-8597-064a5b82b3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:05.161385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df69ba-4eb5-42a0-8597-064a5b82b3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:05.162826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df69ba-4eb5-42a0-8597-064a5b82b3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:05.164981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df69ba-4eb5-42a0-8597-064a5b82b3a7 admin admin] PATH_INFO: `/v3/users/5a5ffd70564f40beafd60758617d3cc9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:05.167760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df69ba-4eb5-42a0-8597-064a5b82b3a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5ffd70564f40beafd60758617d3cc9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:05.179299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df69ba-4eb5-42a0-8597-064a5b82b3a7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:05.357082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-25df69ba-4eb5-42a0-8597-064a5b82b3a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5ffd70564f40beafd60758617d3cc9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:05.405531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-25df69ba-4eb5-42a0-8597-064a5b82b3a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5ffd70564f40beafd60758617d3cc9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:05.407767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-25df69ba-4eb5-42a0-8597-064a5b82b3a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5ffd70564f40beafd60758617d3cc9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:05.408969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-25df69ba-4eb5-42a0-8597-064a5b82b3a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5ffd70564f40beafd60758617d3cc9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:05.427472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-25df69ba-4eb5-42a0-8597-064a5b82b3a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5ffd70564f40beafd60758617d3cc9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:05.478373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 868/1754] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:04 2022] DELETE /identity/v3/users/5a5ffd70564f40beafd60758617d3cc9 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:05.497523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:05.501028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:05.537852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae4af8-129f-41c4-9506-e7c5d28d56a3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:05.763943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:05.765236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:05.768564 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:05.769447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] PATH_INFO: `/v3/users/c7d1c847e29f4272ab79bef320d32c9a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:05.772275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7d1c847e29f4272ab79bef320d32c9a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:05.776339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:05.890950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7d1c847e29f4272ab79bef320d32c9a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:05.892366 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7d1c847e29f4272ab79bef320d32c9a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:05.941787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7d1c847e29f4272ab79bef320d32c9a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:05.962022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7d1c847e29f4272ab79bef320d32c9a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:05.976688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7d1c847e29f4272ab79bef320d32c9a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:05.996559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 887/1755] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:05 2022] DELETE /identity/v3/users/c7d1c847e29f4272ab79bef320d32c9a => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:06.001318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba7922-ab4d-49b1-9bea-553cad00e052 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:06.001655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba7922-ab4d-49b1-9bea-553cad00e052 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:06.001940 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba7922-ab4d-49b1-9bea-553cad00e052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:06.051816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c2ba7922-ab4d-49b1-9bea-553cad00e052 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:06.055859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c2ba7922-ab4d-49b1-9bea-553cad00e052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:06.287086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 869/1756] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:06.351483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:06.354219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:06.414709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5e441-98d8-4452-8bbb-0bb0853ab79c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:06.415236 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5e441-98d8-4452-8bbb-0bb0853ab79c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:06.415659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5e441-98d8-4452-8bbb-0bb0853ab79c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:06.417403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 870/1757] 10.222.0.86 () {54 vars in 921 bytes} [Mon Aug 29 19:02:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:06.429060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b45317-6961-4229-89d1-b37b2bc407f1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:06.429430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b45317-6961-4229-89d1-b37b2bc407f1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:06.429714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b45317-6961-4229-89d1-b37b2bc407f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:06.505244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-52b45317-6961-4229-89d1-b37b2bc407f1 None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:06.512108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-52b45317-6961-4229-89d1-b37b2bc407f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:06.529894 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874c01d-b5a9-40a2-856f-d9dcb2c396f7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:06.544977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742b9a3-dc6c-48bc-a642-1c1cc73cb273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:06.547148 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742b9a3-dc6c-48bc-a642-1c1cc73cb273 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:06.548197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742b9a3-dc6c-48bc-a642-1c1cc73cb273 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:06.548773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742b9a3-dc6c-48bc-a642-1c1cc73cb273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:06.561361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:06.565428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742b9a3-dc6c-48bc-a642-1c1cc73cb273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:06.568916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742b9a3-dc6c-48bc-a642-1c1cc73cb273 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:06.607181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:06.616199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 888/1758] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:02:06.617621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742b9a3-dc6c-48bc-a642-1c1cc73cb273 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:06.630236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6742b9a3-dc6c-48bc-a642-1c1cc73cb273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:06.701374 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742b9a3-dc6c-48bc-a642-1c1cc73cb273 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:06.730799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:06.731371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:06.731655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:06.734864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] PATH_INFO: `/v3/projects/0b66b8cd296344679f5f027db4a847fa` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:06.750937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b66b8cd296344679f5f027db4a847fa)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:06.759329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:06.789919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 871/1759] 10.222.0.86 () {58 vars in 1018 bytes} [Mon Aug 29 19:02:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:06.958645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b66b8cd296344679f5f027db4a847fa', 'request_id': 'req-b71ccc2b-c207-4a8d-9841-94be368b02a5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:06.979490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b66b8cd296344679f5f027db4a847fa', 'request_id': 'req-b71ccc2b-c207-4a8d-9841-94be368b02a5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:06.998887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b66b8cd296344679f5f027db4a847fa', 'request_id': 'req-b71ccc2b-c207-4a8d-9841-94be368b02a5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:07.029144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] The token cache is being invalidate because project 0b66b8cd296344679f5f027db4a847fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:07.029712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:07.030559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:07.034285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 889/1760] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:06 2022] DELETE /identity/v3/projects/0b66b8cd296344679f5f027db4a847fa => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:07.038380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b45317-6961-4229-89d1-b37b2bc407f1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:07.065377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b45317-6961-4229-89d1-b37b2bc407f1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:07.330490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efa422-3714-4771-9932-167842c93836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:07.331555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efa422-3714-4771-9932-167842c93836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:07.331555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efa422-3714-4771-9932-167842c93836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:07.335587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efa422-3714-4771-9932-167842c93836 admin admin] PATH_INFO: `/v3/users/4dbced0836e649d99d358dc555da87e8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:07.337052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efa422-3714-4771-9932-167842c93836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dbced0836e649d99d358dc555da87e8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:07.342786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efa422-3714-4771-9932-167842c93836 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:07.466166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-06efa422-3714-4771-9932-167842c93836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dbced0836e649d99d358dc555da87e8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:07.520567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-06efa422-3714-4771-9932-167842c93836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dbced0836e649d99d358dc555da87e8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:07.521566 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-06efa422-3714-4771-9932-167842c93836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dbced0836e649d99d358dc555da87e8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:07.522666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-06efa422-3714-4771-9932-167842c93836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dbced0836e649d99d358dc555da87e8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:07.549375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-06efa422-3714-4771-9932-167842c93836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dbced0836e649d99d358dc555da87e8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:07.577768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 872/1761] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:07 2022] DELETE /identity/v3/users/4dbced0836e649d99d358dc555da87e8 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:07.928515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:07.930891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:08.103653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ccc2b-c207-4a8d-9841-94be368b02a5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:08.254444 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396b82a-3a9a-49b1-8e84-49ac2129d2f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:08.255886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b82a-3a9a-49b1-8e84-49ac2129d2f2 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:08.256470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b82a-3a9a-49b1-8e84-49ac2129d2f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:08.257786 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b82a-3a9a-49b1-8e84-49ac2129d2f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:08.261959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a396b82a-3a9a-49b1-8e84-49ac2129d2f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:08.309504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396b82a-3a9a-49b1-8e84-49ac2129d2f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:08.311774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396b82a-3a9a-49b1-8e84-49ac2129d2f2 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:08.348401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 890/1762] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:08.622049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efa422-3714-4771-9932-167842c93836 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:08.623688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-06efa422-3714-4771-9932-167842c93836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:08.836992 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:08.964779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efa422-3714-4771-9932-167842c93836 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:09.003733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:09.005928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:09.007805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:09.011556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] PATH_INFO: `/v3/projects/140c9c3127124791acd2c9267aac62a5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:09.017713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] RBAC: Authorizing `identity:delete_project(project_id=140c9c3127124791acd2c9267aac62a5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:09.028006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:09.222590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '140c9c3127124791acd2c9267aac62a5', 'request_id': 'req-1ecc442d-308c-452e-ab5e-a5151906613c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:09.248058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '140c9c3127124791acd2c9267aac62a5', 'request_id': 'req-1ecc442d-308c-452e-ab5e-a5151906613c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:09.269605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '140c9c3127124791acd2c9267aac62a5', 'request_id': 'req-1ecc442d-308c-452e-ab5e-a5151906613c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:09.318795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] The token cache is being invalidate because project 140c9c3127124791acd2c9267aac62a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:09.319153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:09.326083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ecc442d-308c-452e-ab5e-a5151906613c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:09.330784 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 873/1763] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:08 2022] DELETE /identity/v3/projects/140c9c3127124791acd2c9267aac62a5 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:09.337016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b380-a2d5-40a7-92ab-40d5bfab7812 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:09.337615 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b380-a2d5-40a7-92ab-40d5bfab7812 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:09.337887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b380-a2d5-40a7-92ab-40d5bfab7812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:09.367485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:09.420519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8802b380-a2d5-40a7-92ab-40d5bfab7812 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:09.427867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8802b380-a2d5-40a7-92ab-40d5bfab7812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:09.636617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 891/1764] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:09.646531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c64ee4-599c-4c53-98ab-f6fba3a4d18a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:09.647472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c64ee4-599c-4c53-98ab-f6fba3a4d18a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:09.648256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c64ee4-599c-4c53-98ab-f6fba3a4d18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:09.727719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b5c64ee4-599c-4c53-98ab-f6fba3a4d18a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:09.732495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b5c64ee4-599c-4c53-98ab-f6fba3a4d18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:09.770778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802b380-a2d5-40a7-92ab-40d5bfab7812 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:09.780290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 874/1765] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:09.785674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802b380-a2d5-40a7-92ab-40d5bfab7812 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:09.800629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e99c8-a507-4733-bbc2-b99f13679c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:09.801696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e99c8-a507-4733-bbc2-b99f13679c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:09.802865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e99c8-a507-4733-bbc2-b99f13679c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:09.803403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e99c8-a507-4733-bbc2-b99f13679c78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:09.804319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e99c8-a507-4733-bbc2-b99f13679c78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:09.804878 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e99c8-a507-4733-bbc2-b99f13679c78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:09.807191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e99c8-a507-4733-bbc2-b99f13679c78 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:09.837017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 892/1766] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:02:09.841298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c64ee4-599c-4c53-98ab-f6fba3a4d18a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:09.858980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c64ee4-599c-4c53-98ab-f6fba3a4d18a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:09.872683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf63be-40f4-4f1d-a607-cffb7741cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:09.873615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf63be-40f4-4f1d-a607-cffb7741cfce admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:09.874143 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf63be-40f4-4f1d-a607-cffb7741cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:09.874692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf63be-40f4-4f1d-a607-cffb7741cfce admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:09.875752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf63be-40f4-4f1d-a607-cffb7741cfce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:09.879029 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf63be-40f4-4f1d-a607-cffb7741cfce admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:09.951991 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 875/1767] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:09 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:09.957492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e99c8-a507-4733-bbc2-b99f13679c78 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:09.976650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e99c8-a507-4733-bbc2-b99f13679c78 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:10.146351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856606f-3f78-4903-9935-936b04834a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:10.147960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856606f-3f78-4903-9935-936b04834a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:10.149560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856606f-3f78-4903-9935-936b04834a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:10.150816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856606f-3f78-4903-9935-936b04834a7f admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:10.162027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856606f-3f78-4903-9935-936b04834a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:10.166547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856606f-3f78-4903-9935-936b04834a7f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:10.188219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf63be-40f4-4f1d-a607-cffb7741cfce admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:10.195676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-aeaf63be-40f4-4f1d-a607-cffb7741cfce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:10.235588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 893/1768] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:09 2022] POST /identity/v3/users => generated 291 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:10.235588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856606f-3f78-4903-9935-936b04834a7f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:10.244857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856606f-3f78-4903-9935-936b04834a7f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:10.248920 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf63be-40f4-4f1d-a607-cffb7741cfce admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:10.257988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:10.258675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:10.258957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:10.259265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] PATH_INFO: `/v3/projects/e951777ddc97490f84efc416446b9945` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:10.260571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e951777ddc97490f84efc416446b9945)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:10.262937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:10.270779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a4aa1-d09d-45d2-95da-1882358770ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:10.272728 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4aa1-d09d-45d2-95da-1882358770ad admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:10.273097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4aa1-d09d-45d2-95da-1882358770ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:10.275066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4aa1-d09d-45d2-95da-1882358770ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:10.275962 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a4aa1-d09d-45d2-95da-1882358770ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:10.280480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a4aa1-d09d-45d2-95da-1882358770ad admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:10.300378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 894/1769] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:10.301612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a4aa1-d09d-45d2-95da-1882358770ad admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:10.325563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a4aa1-d09d-45d2-95da-1882358770ad admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:10.345056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ad5ee-0f46-4e01-97f6-e07b46f9664e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:10.346307 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ad5ee-0f46-4e01-97f6-e07b46f9664e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:10.346816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ad5ee-0f46-4e01-97f6-e07b46f9664e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:10.347387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ad5ee-0f46-4e01-97f6-e07b46f9664e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:10.348441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ad5ee-0f46-4e01-97f6-e07b46f9664e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:10.350552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ad5ee-0f46-4e01-97f6-e07b46f9664e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:10.370636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 895/1770] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:10.371935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ad5ee-0f46-4e01-97f6-e07b46f9664e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:10.388306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ad5ee-0f46-4e01-97f6-e07b46f9664e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:10.394457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e951777ddc97490f84efc416446b9945', 'request_id': 'req-5f89bc62-b226-44aa-8041-57b9a295a089'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:10.406538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e951777ddc97490f84efc416446b9945', 'request_id': 'req-5f89bc62-b226-44aa-8041-57b9a295a089'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:10.415160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e951777ddc97490f84efc416446b9945', 'request_id': 'req-5f89bc62-b226-44aa-8041-57b9a295a089'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:10.469320 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] The token cache is being invalidate because project e951777ddc97490f84efc416446b9945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:10.474355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:10.480765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:10.493709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 876/1771] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:10 2022] DELETE /identity/v3/projects/e951777ddc97490f84efc416446b9945 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:10.514990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:10.517539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:10.524584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:10.551343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082a0a2-9f24-4e73-905c-d7efc6ec9336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:10.552491 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082a0a2-9f24-4e73-905c-d7efc6ec9336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:10.552879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082a0a2-9f24-4e73-905c-d7efc6ec9336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:10.553259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082a0a2-9f24-4e73-905c-d7efc6ec9336 admin admin] PATH_INFO: `/v3/projects/d51c4735cda74af58b39986751b12313/users/33974ab4030e4125b89c593fca3e5175/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:10.588009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082a0a2-9f24-4e73-905c-d7efc6ec9336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d51c4735cda74af58b39986751b12313, user_id=33974ab4030e4125b89c593fca3e5175, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:10.592306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082a0a2-9f24-4e73-905c-d7efc6ec9336 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:10.614816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:10.648531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 896/1772] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:10 2022] PUT /identity/v3/projects/d51c4735cda74af58b39986751b12313/users/33974ab4030e4125b89c593fca3e5175/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:10.649672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082a0a2-9f24-4e73-905c-d7efc6ec9336 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:10.656573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9082a0a2-9f24-4e73-905c-d7efc6ec9336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:10.755318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89bc62-b226-44aa-8041-57b9a295a089 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:10.858503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:10.867933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:10.868226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:10.868520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] PATH_INFO: `/v3/projects/8c3428a94007405ca776b211c15375c5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:10.871677 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c3428a94007405ca776b211c15375c5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:10.885115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:10.934211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082a0a2-9f24-4e73-905c-d7efc6ec9336 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:10.948626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba611374-cae0-4126-a90d-5757cafc9e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:10.949570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba611374-cae0-4126-a90d-5757cafc9e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:10.950360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba611374-cae0-4126-a90d-5757cafc9e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:10.955574 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba611374-cae0-4126-a90d-5757cafc9e04 admin admin] PATH_INFO: `/v3/users/dfa76489385f479abd77f0f18faec273` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:10.955574 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba611374-cae0-4126-a90d-5757cafc9e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa76489385f479abd77f0f18faec273)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:10.959966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba611374-cae0-4126-a90d-5757cafc9e04 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:11.045374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c3428a94007405ca776b211c15375c5', 'request_id': 'req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:11.080417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c3428a94007405ca776b211c15375c5', 'request_id': 'req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:11.101678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c3428a94007405ca776b211c15375c5', 'request_id': 'req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:11.144464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] The token cache is being invalidate because project 8c3428a94007405ca776b211c15375c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:11.145918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:11.152492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c9b7cf5-080f-4c88-9065-5ef93aec9ef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:11.154284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba611374-cae0-4126-a90d-5757cafc9e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa76489385f479abd77f0f18faec273'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:11.155195 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba611374-cae0-4126-a90d-5757cafc9e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa76489385f479abd77f0f18faec273'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:11.161654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 877/1773] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:10 2022] DELETE /identity/v3/projects/8c3428a94007405ca776b211c15375c5 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:11.164827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c459e-8592-43d3-ba43-84f49f0c26b0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:11.169473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c459e-8592-43d3-ba43-84f49f0c26b0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:11.172319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c459e-8592-43d3-ba43-84f49f0c26b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:11.219485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba611374-cae0-4126-a90d-5757cafc9e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa76489385f479abd77f0f18faec273'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:11.257393 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba611374-cae0-4126-a90d-5757cafc9e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa76489385f479abd77f0f18faec273'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:11.306062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba611374-cae0-4126-a90d-5757cafc9e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa76489385f479abd77f0f18faec273'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:11.331350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-899c459e-8592-43d3-ba43-84f49f0c26b0 None None] MFA Rules not processed for user `33974ab4030e4125b89c593fca3e5175`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:11.332636 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-899c459e-8592-43d3-ba43-84f49f0c26b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:11.335751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 897/1774] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:10 2022] DELETE /identity/v3/users/dfa76489385f479abd77f0f18faec273 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:11.340499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebb59-e566-4acb-8a0c-9945d2695c38 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:11.340798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebb59-e566-4acb-8a0c-9945d2695c38 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:11.341075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebb59-e566-4acb-8a0c-9945d2695c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:11.425063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-520ebb59-e566-4acb-8a0c-9945d2695c38 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:11.431168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-520ebb59-e566-4acb-8a0c-9945d2695c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:11.525591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:11.588004 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 878/1775] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:11.592255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c459e-8592-43d3-ba43-84f49f0c26b0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:11.603499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-899c459e-8592-43d3-ba43-84f49f0c26b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:11.628197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:11.672887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 898/1776] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:11.687222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792f25-65c5-4d0c-9066-2fce309b4bcd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:11.689548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792f25-65c5-4d0c-9066-2fce309b4bcd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:11.692028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792f25-65c5-4d0c-9066-2fce309b4bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:11.839383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8b792f25-65c5-4d0c-9066-2fce309b4bcd None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:11.849131 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8b792f25-65c5-4d0c-9066-2fce309b4bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:11.886608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c459e-8592-43d3-ba43-84f49f0c26b0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:11.893490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 899/1777] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:11.899178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49701d9d-97f9-455d-829d-8dab3d65b9be None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:11.900448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49701d9d-97f9-455d-829d-8dab3d65b9be None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:11.902924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649c47-f06c-44e5-a8cf-41edbb364c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:11.904013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649c47-f06c-44e5-a8cf-41edbb364c9d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:11.904474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649c47-f06c-44e5-a8cf-41edbb364c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:11.904797 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49701d9d-97f9-455d-829d-8dab3d65b9be None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:11.905047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649c47-f06c-44e5-a8cf-41edbb364c9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:11.907794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95649c47-f06c-44e5-a8cf-41edbb364c9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:11.911985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95649c47-f06c-44e5-a8cf-41edbb364c9d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:11.915843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 900/1778] 10.222.0.86 () {54 vars in 921 bytes} [Mon Aug 29 19:02:11 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:11.923715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49701d9d-97f9-455d-829d-8dab3d65b9be None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:11.985009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49701d9d-97f9-455d-829d-8dab3d65b9be None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:11.989938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 879/1779] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:11 2022] POST /identity/v3/projects => generated 383 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:12.002675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649c47-f06c-44e5-a8cf-41edbb364c9d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:12.003760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2f83b-0ecf-4201-a77e-3a5b365b1af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:12.005030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2f83b-0ecf-4201-a77e-3a5b365b1af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.009693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2f83b-0ecf-4201-a77e-3a5b365b1af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.011387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2f83b-0ecf-4201-a77e-3a5b365b1af6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.016999 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2f83b-0ecf-4201-a77e-3a5b365b1af6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:12.017721 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2f83b-0ecf-4201-a77e-3a5b365b1af6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:12.028143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2f83b-0ecf-4201-a77e-3a5b365b1af6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:12.110479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 901/1780] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:12.114489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2f83b-0ecf-4201-a77e-3a5b365b1af6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:12.141876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649c47-f06c-44e5-a8cf-41edbb364c9d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:12.159360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c903b6-492c-4a2e-8c58-6f4c30b34ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:12.160059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c903b6-492c-4a2e-8c58-6f4c30b34ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.160347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c903b6-492c-4a2e-8c58-6f4c30b34ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.160570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c903b6-492c-4a2e-8c58-6f4c30b34ea0 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.161406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c903b6-492c-4a2e-8c58-6f4c30b34ea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:12.164184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c903b6-492c-4a2e-8c58-6f4c30b34ea0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:12.169202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2f83b-0ecf-4201-a77e-3a5b365b1af6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:12.187290 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb839c-723d-4cc8-bad4-d7733494e23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:12.188007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb839c-723d-4cc8-bad4-d7733494e23f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.192804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb839c-723d-4cc8-bad4-d7733494e23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.193310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb839c-723d-4cc8-bad4-d7733494e23f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.194516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb839c-723d-4cc8-bad4-d7733494e23f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:12.195248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 880/1781] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:12 2022] POST /identity/v3/users => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:12.206878 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb839c-723d-4cc8-bad4-d7733494e23f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:12.209229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c903b6-492c-4a2e-8c58-6f4c30b34ea0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:12.248326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c903b6-492c-4a2e-8c58-6f4c30b34ea0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:12.263403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5c6d7-8439-4148-9c7f-b9a0daac628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:12.266161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5c6d7-8439-4148-9c7f-b9a0daac628d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.266555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5c6d7-8439-4148-9c7f-b9a0daac628d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.267059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5c6d7-8439-4148-9c7f-b9a0daac628d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.267800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5c6d7-8439-4148-9c7f-b9a0daac628d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:12.281689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5c6d7-8439-4148-9c7f-b9a0daac628d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:12.307216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 881/1782] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:12.309060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c27aa-1f77-43f9-b3fb-5d5bb1dd1f0f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.309357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c27aa-1f77-43f9-b3fb-5d5bb1dd1f0f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.309700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c27aa-1f77-43f9-b3fb-5d5bb1dd1f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.377650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5f5c27aa-1f77-43f9-b3fb-5d5bb1dd1f0f None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:12.378174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 902/1783] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:12 2022] POST /identity/v3/projects => generated 373 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:12.380103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5f5c27aa-1f77-43f9-b3fb-5d5bb1dd1f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:12.383496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb839c-723d-4cc8-bad4-d7733494e23f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:12.533637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:12.568893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 882/1784] 10.222.0.86 () {58 vars in 1018 bytes} [Mon Aug 29 19:02:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:12.568893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c27aa-1f77-43f9-b3fb-5d5bb1dd1f0f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:12.678477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c27aa-1f77-43f9-b3fb-5d5bb1dd1f0f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:12.686385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39f4c2-0f52-45e3-aec2-c75499719797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:12.688610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f4c2-0f52-45e3-aec2-c75499719797 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.689084 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f4c2-0f52-45e3-aec2-c75499719797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.689408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f4c2-0f52-45e3-aec2-c75499719797 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.690062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39f4c2-0f52-45e3-aec2-c75499719797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:12.692037 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39f4c2-0f52-45e3-aec2-c75499719797 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:12.696086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb839c-723d-4cc8-bad4-d7733494e23f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:12.708718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830ece5-2838-4832-9361-38ca890ab848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:12.709422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830ece5-2838-4832-9361-38ca890ab848 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.709770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830ece5-2838-4832-9361-38ca890ab848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.710248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830ece5-2838-4832-9361-38ca890ab848 admin admin] PATH_INFO: `/v3/projects/35bd38302d9d41bfbb9b9400ceeb29a6/users/2208d58795ff44b4b5ca08d7397b5983/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.753814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 883/1785] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:12 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:12.758181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39f4c2-0f52-45e3-aec2-c75499719797 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:12.766014 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830ece5-2838-4832-9361-38ca890ab848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35bd38302d9d41bfbb9b9400ceeb29a6, user_id=2208d58795ff44b4b5ca08d7397b5983, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:12.772168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830ece5-2838-4832-9361-38ca890ab848 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:12.780256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39f4c2-0f52-45e3-aec2-c75499719797 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:12.789097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff16b22-8748-4cce-a7e4-0dca9654903d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:12.790077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff16b22-8748-4cce-a7e4-0dca9654903d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.790350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff16b22-8748-4cce-a7e4-0dca9654903d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.790907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff16b22-8748-4cce-a7e4-0dca9654903d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.791923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff16b22-8748-4cce-a7e4-0dca9654903d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:12.793696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff16b22-8748-4cce-a7e4-0dca9654903d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:12.806172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 903/1786] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:12 2022] PUT /identity/v3/projects/35bd38302d9d41bfbb9b9400ceeb29a6/users/2208d58795ff44b4b5ca08d7397b5983/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:12.811016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830ece5-2838-4832-9361-38ca890ab848 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:12.841932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 884/1787] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:12.843528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff16b22-8748-4cce-a7e4-0dca9654903d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:12.868407 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff16b22-8748-4cce-a7e4-0dca9654903d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:12.977479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830ece5-2838-4832-9361-38ca890ab848 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:12.990972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92c8a1-3216-4398-a166-3c1be4782eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:12.991843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92c8a1-3216-4398-a166-3c1be4782eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:12.992131 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92c8a1-3216-4398-a166-3c1be4782eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:12.992409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92c8a1-3216-4398-a166-3c1be4782eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:12.993078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92c8a1-3216-4398-a166-3c1be4782eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:12.994677 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92c8a1-3216-4398-a166-3c1be4782eb5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:13.012029 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78639fd-4a64-48da-9f37-6da5a59fdc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:13.021586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78639fd-4a64-48da-9f37-6da5a59fdc74 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:13.021586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78639fd-4a64-48da-9f37-6da5a59fdc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:13.021586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78639fd-4a64-48da-9f37-6da5a59fdc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:13.021586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78639fd-4a64-48da-9f37-6da5a59fdc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:13.021586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78639fd-4a64-48da-9f37-6da5a59fdc74 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:13.025057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 904/1788] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:13.025970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92c8a1-3216-4398-a166-3c1be4782eb5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:13.045739 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 885/1789] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:13.045964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92c8a1-3216-4398-a166-3c1be4782eb5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:13.051971 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78639fd-4a64-48da-9f37-6da5a59fdc74 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:13.069228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78639fd-4a64-48da-9f37-6da5a59fdc74 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:13.069505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fbe15-3e80-45dc-b7ab-28728736effe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:13.071452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fbe15-3e80-45dc-b7ab-28728736effe admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:13.071741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fbe15-3e80-45dc-b7ab-28728736effe admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:13.072026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fbe15-3e80-45dc-b7ab-28728736effe admin admin] PATH_INFO: `/v3/projects/35bd38302d9d41bfbb9b9400ceeb29a6/users/2208d58795ff44b4b5ca08d7397b5983/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:13.077532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fbe15-3e80-45dc-b7ab-28728736effe admin admin] RBAC: Authorizing `identity:create_grant(project_id=35bd38302d9d41bfbb9b9400ceeb29a6, user_id=2208d58795ff44b4b5ca08d7397b5983, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:13.083893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fbe15-3e80-45dc-b7ab-28728736effe admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:13.085371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2945349-96fc-4df9-8cb2-0e3b0118cc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:13.086026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945349-96fc-4df9-8cb2-0e3b0118cc77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:13.086512 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945349-96fc-4df9-8cb2-0e3b0118cc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:13.086863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945349-96fc-4df9-8cb2-0e3b0118cc77 admin admin] PATH_INFO: `/v3/projects/bf2a001e8e6a4f7d8c8d7a49ad6eae6d/users/fa8c72b69f3f45e49eef5fa9b8f60f05/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:13.118259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 905/1790] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:13 2022] PUT /identity/v3/projects/35bd38302d9d41bfbb9b9400ceeb29a6/users/2208d58795ff44b4b5ca08d7397b5983/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:13.125036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2945349-96fc-4df9-8cb2-0e3b0118cc77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf2a001e8e6a4f7d8c8d7a49ad6eae6d, user_id=fa8c72b69f3f45e49eef5fa9b8f60f05, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:13.125934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2945349-96fc-4df9-8cb2-0e3b0118cc77 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:13.130548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20acb15-b72b-4e58-9c2f-6d426c4d25e3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:13.130858 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20acb15-b72b-4e58-9c2f-6d426c4d25e3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:13.131411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20acb15-b72b-4e58-9c2f-6d426c4d25e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:13.165699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 886/1791] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:13 2022] PUT /identity/v3/projects/bf2a001e8e6a4f7d8c8d7a49ad6eae6d/users/fa8c72b69f3f45e49eef5fa9b8f60f05/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:13.181868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe61131-4d5e-465d-ac47-c18c46e96def None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:13.182504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe61131-4d5e-465d-ac47-c18c46e96def None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:13.182669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe61131-4d5e-465d-ac47-c18c46e96def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:13.222914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f20acb15-b72b-4e58-9c2f-6d426c4d25e3 None None] MFA Rules not processed for user `2208d58795ff44b4b5ca08d7397b5983`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:13.224396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f20acb15-b72b-4e58-9c2f-6d426c4d25e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:13.262303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:13.264241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2fe61131-4d5e-465d-ac47-c18c46e96def None None] MFA Rules not processed for user `fa8c72b69f3f45e49eef5fa9b8f60f05`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:13.272966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2fe61131-4d5e-465d-ac47-c18c46e96def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:13.415091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:13.433876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 906/1792] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:13.446910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20acb15-b72b-4e58-9c2f-6d426c4d25e3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:13.451666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f20acb15-b72b-4e58-9c2f-6d426c4d25e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:13.543343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 887/1793] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:13.547998 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe61131-4d5e-465d-ac47-c18c46e96def None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:13.645988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20acb15-b72b-4e58-9c2f-6d426c4d25e3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:13.656269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9c188-f89a-48ba-8089-74de4b74ab31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:13.659325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9c188-f89a-48ba-8089-74de4b74ab31 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:13.659725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9c188-f89a-48ba-8089-74de4b74ab31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:13.660093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9c188-f89a-48ba-8089-74de4b74ab31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:13.696918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da9c188-f89a-48ba-8089-74de4b74ab31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:13.699559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da9c188-f89a-48ba-8089-74de4b74ab31 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:13.731778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 907/1794] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:13.732751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9c188-f89a-48ba-8089-74de4b74ab31 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:13.740642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6da9c188-f89a-48ba-8089-74de4b74ab31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:13.838393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe61131-4d5e-465d-ac47-c18c46e96def None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:13.852868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3f0b-5a6f-4432-849a-81028615f658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:13.853665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3f0b-5a6f-4432-849a-81028615f658 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:13.854056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3f0b-5a6f-4432-849a-81028615f658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:13.854404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3f0b-5a6f-4432-849a-81028615f658 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:13.855590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e3f0b-5a6f-4432-849a-81028615f658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:13.860010 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e3f0b-5a6f-4432-849a-81028615f658 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:13.927570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 888/1795] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:13 2022] POST /identity/v3/projects => generated 369 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:13.936358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3f0b-5a6f-4432-849a-81028615f658 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:14.024245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9c188-f89a-48ba-8089-74de4b74ab31 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:14.154014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3f0b-5a6f-4432-849a-81028615f658 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:14.168314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:14.177927 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93756c98-f66f-4ddb-94e3-26400554f92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:14.179267 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93756c98-f66f-4ddb-94e3-26400554f92a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:14.179570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93756c98-f66f-4ddb-94e3-26400554f92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:14.179861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93756c98-f66f-4ddb-94e3-26400554f92a admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:14.180706 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93756c98-f66f-4ddb-94e3-26400554f92a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:14.182758 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93756c98-f66f-4ddb-94e3-26400554f92a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:14.231792 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42cdef-bc5a-4171-9b89-cf26eeb171b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:14.233913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42cdef-bc5a-4171-9b89-cf26eeb171b4 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:14.235067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42cdef-bc5a-4171-9b89-cf26eeb171b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:14.236154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42cdef-bc5a-4171-9b89-cf26eeb171b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:14.248366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 889/1796] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:13 2022] POST /identity/v3/users => generated 291 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:14.249581 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93756c98-f66f-4ddb-94e3-26400554f92a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:14.258162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42cdef-bc5a-4171-9b89-cf26eeb171b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:14.259867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42cdef-bc5a-4171-9b89-cf26eeb171b4 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:14.294260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 908/1797] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:14.295247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42cdef-bc5a-4171-9b89-cf26eeb171b4 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:14.319825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42cdef-bc5a-4171-9b89-cf26eeb171b4 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:14.332578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b26cd-ee4c-46c4-bdc1-e35db5fe4057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:14.333204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b26cd-ee4c-46c4-bdc1-e35db5fe4057 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:14.333487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b26cd-ee4c-46c4-bdc1-e35db5fe4057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:14.333760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b26cd-ee4c-46c4-bdc1-e35db5fe4057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:14.334406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b26cd-ee4c-46c4-bdc1-e35db5fe4057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:14.340357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b26cd-ee4c-46c4-bdc1-e35db5fe4057 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:14.366592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 909/1798] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:14.367568 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b26cd-ee4c-46c4-bdc1-e35db5fe4057 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:14.391181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b26cd-ee4c-46c4-bdc1-e35db5fe4057 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:14.410589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395f3a0-12c6-4bbc-9c4e-0732539c2505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:14.411311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395f3a0-12c6-4bbc-9c4e-0732539c2505 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:14.411589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395f3a0-12c6-4bbc-9c4e-0732539c2505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:14.411864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395f3a0-12c6-4bbc-9c4e-0732539c2505 admin admin] PATH_INFO: `/v3/projects/494f3a99672747f99f55ee7ec723d2e4/users/fdb0e67776374cee97ffad0b0d27070b/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:14.457396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395f3a0-12c6-4bbc-9c4e-0732539c2505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=494f3a99672747f99f55ee7ec723d2e4, user_id=fdb0e67776374cee97ffad0b0d27070b, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:14.458071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:14.470176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395f3a0-12c6-4bbc-9c4e-0732539c2505 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:14.489612 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93756c98-f66f-4ddb-94e3-26400554f92a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:14.501382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6b0ae-840e-4d13-a087-61b6b972b2e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:14.503365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6b0ae-840e-4d13-a087-61b6b972b2e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:14.507561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6b0ae-840e-4d13-a087-61b6b972b2e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:14.511171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6b0ae-840e-4d13-a087-61b6b972b2e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:14.519283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 910/1799] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:14 2022] PUT /identity/v3/projects/494f3a99672747f99f55ee7ec723d2e4/users/fdb0e67776374cee97ffad0b0d27070b/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:14.520916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395f3a0-12c6-4bbc-9c4e-0732539c2505 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:14.624849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6b0ae-840e-4d13-a087-61b6b972b2e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:14.627427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6b0ae-840e-4d13-a087-61b6b972b2e3 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:14.643098 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 890/1800] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:14.674984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395f3a0-12c6-4bbc-9c4e-0732539c2505 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:14.692461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2272662-ddd1-4d71-bda9-5620d30c0155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:14.693352 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2272662-ddd1-4d71-bda9-5620d30c0155 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:14.693630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2272662-ddd1-4d71-bda9-5620d30c0155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:14.693899 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2272662-ddd1-4d71-bda9-5620d30c0155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:14.694550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2272662-ddd1-4d71-bda9-5620d30c0155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:14.696187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2272662-ddd1-4d71-bda9-5620d30c0155 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:14.719062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 911/1801] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:14.723992 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6b0ae-840e-4d13-a087-61b6b972b2e3 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:14.745913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6b0ae-840e-4d13-a087-61b6b972b2e3 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:14.760780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2a939-49a9-41d5-b8ef-454d30279d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:14.762951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2a939-49a9-41d5-b8ef-454d30279d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:14.763479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2a939-49a9-41d5-b8ef-454d30279d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:14.764661 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2a939-49a9-41d5-b8ef-454d30279d8c admin admin] PATH_INFO: `/v3/projects/494f3a99672747f99f55ee7ec723d2e4/users/fdb0e67776374cee97ffad0b0d27070b/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:14.771441 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2a939-49a9-41d5-b8ef-454d30279d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=494f3a99672747f99f55ee7ec723d2e4, user_id=fdb0e67776374cee97ffad0b0d27070b, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:14.781388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2a939-49a9-41d5-b8ef-454d30279d8c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:14.821740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 891/1802] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:14 2022] PUT /identity/v3/projects/494f3a99672747f99f55ee7ec723d2e4/users/fdb0e67776374cee97ffad0b0d27070b/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:14.823695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163ce62-ac11-4faf-b3ac-bb98cdb2e523 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:14.823986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163ce62-ac11-4faf-b3ac-bb98cdb2e523 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:14.824267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163ce62-ac11-4faf-b3ac-bb98cdb2e523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:14.876401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5163ce62-ac11-4faf-b3ac-bb98cdb2e523 None None] MFA Rules not processed for user `fdb0e67776374cee97ffad0b0d27070b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:14.877634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5163ce62-ac11-4faf-b3ac-bb98cdb2e523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:14.976015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2a939-49a9-41d5-b8ef-454d30279d8c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:14.976436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 912/1803] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:14.977772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-83b2a939-49a9-41d5-b8ef-454d30279d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:15.082984 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2a939-49a9-41d5-b8ef-454d30279d8c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.087619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9bdbc-c9c9-4dbf-90b6-0333fa835e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.088246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9bdbc-c9c9-4dbf-90b6-0333fa835e85 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.098074 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9bdbc-c9c9-4dbf-90b6-0333fa835e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.098360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9bdbc-c9c9-4dbf-90b6-0333fa835e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.124395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9bdbc-c9c9-4dbf-90b6-0333fa835e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.145717 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9bdbc-c9c9-4dbf-90b6-0333fa835e85 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.185137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 892/1804] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:15.214110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163ce62-ac11-4faf-b3ac-bb98cdb2e523 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.215758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5163ce62-ac11-4faf-b3ac-bb98cdb2e523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:15.278628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:15.314082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9bdbc-c9c9-4dbf-90b6-0333fa835e85 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.325907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9bdbc-c9c9-4dbf-90b6-0333fa835e85 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.344235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15b989-4a4d-41b2-ac51-9e62499c7689 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.348626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15b989-4a4d-41b2-ac51-9e62499c7689 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.348626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15b989-4a4d-41b2-ac51-9e62499c7689 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.348916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15b989-4a4d-41b2-ac51-9e62499c7689 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.364325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163ce62-ac11-4faf-b3ac-bb98cdb2e523 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.372954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.374134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.375547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.376125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] PATH_INFO: `/v3/projects/bf137add5a2245488c6a55cafa2ea143` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.378079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf137add5a2245488c6a55cafa2ea143)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.380373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.428458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15b989-4a4d-41b2-ac51-9e62499c7689 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.430373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15b989-4a4d-41b2-ac51-9e62499c7689 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.442969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 893/1805] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:02:15.443302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf137add5a2245488c6a55cafa2ea143', 'request_id': 'req-d7b75f5e-e449-419d-b892-b38900605e52'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:15.454556 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf137add5a2245488c6a55cafa2ea143', 'request_id': 'req-d7b75f5e-e449-419d-b892-b38900605e52'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:15.461951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf137add5a2245488c6a55cafa2ea143', 'request_id': 'req-d7b75f5e-e449-419d-b892-b38900605e52'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:15.475658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] The token cache is being invalidate because project bf137add5a2245488c6a55cafa2ea143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:15.476019 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:15.476739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d7b75f5e-e449-419d-b892-b38900605e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:15.478532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 913/1806] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:15 2022] DELETE /identity/v3/projects/bf137add5a2245488c6a55cafa2ea143 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:15.483296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285b69c-37a5-4325-9db4-cea962c0050e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.483760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285b69c-37a5-4325-9db4-cea962c0050e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.484020 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285b69c-37a5-4325-9db4-cea962c0050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.506672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7285b69c-37a5-4325-9db4-cea962c0050e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:15.508211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7285b69c-37a5-4325-9db4-cea962c0050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:15.537871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:15.605813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 894/1807] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:15.613885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d616-4ec5-4e11-983d-63856345dc65 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.614174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d616-4ec5-4e11-983d-63856345dc65 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.614471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d616-4ec5-4e11-983d-63856345dc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.635446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-55c9d616-4ec5-4e11-983d-63856345dc65 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:15.636704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-55c9d616-4ec5-4e11-983d-63856345dc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:15.645000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 914/1808] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:15.649267 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285b69c-37a5-4325-9db4-cea962c0050e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.659617 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285b69c-37a5-4325-9db4-cea962c0050e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.664819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b047d-fd28-4f1a-a610-08ce955a71e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.665430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b047d-fd28-4f1a-a610-08ce955a71e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.665709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b047d-fd28-4f1a-a610-08ce955a71e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.666004 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b047d-fd28-4f1a-a610-08ce955a71e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.666653 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b047d-fd28-4f1a-a610-08ce955a71e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:15.666951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b047d-fd28-4f1a-a610-08ce955a71e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.668572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b047d-fd28-4f1a-a610-08ce955a71e3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.686464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 895/1809] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:15.691469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9d616-4ec5-4e11-983d-63856345dc65 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.700968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9d616-4ec5-4e11-983d-63856345dc65 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.705881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38decead-8a74-40db-9bda-9f0d27eeb268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.706451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38decead-8a74-40db-9bda-9f0d27eeb268 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.706732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38decead-8a74-40db-9bda-9f0d27eeb268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.707024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38decead-8a74-40db-9bda-9f0d27eeb268 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.707672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38decead-8a74-40db-9bda-9f0d27eeb268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.709543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38decead-8a74-40db-9bda-9f0d27eeb268 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.736225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 915/1810] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:15 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:15.740359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b047d-fd28-4f1a-a610-08ce955a71e3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.751115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b047d-fd28-4f1a-a610-08ce955a71e3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.807794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763988d7-92bb-448f-8225-fa1c5976526c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.808322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763988d7-92bb-448f-8225-fa1c5976526c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.808603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763988d7-92bb-448f-8225-fa1c5976526c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.808877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763988d7-92bb-448f-8225-fa1c5976526c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.809543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763988d7-92bb-448f-8225-fa1c5976526c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.811479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763988d7-92bb-448f-8225-fa1c5976526c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.834023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 896/1811] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:15 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:15.838634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38decead-8a74-40db-9bda-9f0d27eeb268 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.850505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38decead-8a74-40db-9bda-9f0d27eeb268 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.855095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdff4c3-e55c-40c2-a815-bad89878e0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.855809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdff4c3-e55c-40c2-a815-bad89878e0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.865320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdff4c3-e55c-40c2-a815-bad89878e0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.865621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdff4c3-e55c-40c2-a815-bad89878e0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.866309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdff4c3-e55c-40c2-a815-bad89878e0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.867936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdff4c3-e55c-40c2-a815-bad89878e0d6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.883044 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 916/1812] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:15.886003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763988d7-92bb-448f-8225-fa1c5976526c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.899146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763988d7-92bb-448f-8225-fa1c5976526c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.903956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faada6e-e1c1-4c82-a579-93e59105cfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.904564 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faada6e-e1c1-4c82-a579-93e59105cfad admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.904844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faada6e-e1c1-4c82-a579-93e59105cfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.905188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faada6e-e1c1-4c82-a579-93e59105cfad admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.905797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faada6e-e1c1-4c82-a579-93e59105cfad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.907979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faada6e-e1c1-4c82-a579-93e59105cfad admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.926604 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 897/1813] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:15.927623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdff4c3-e55c-40c2-a815-bad89878e0d6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:15.939266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdff4c3-e55c-40c2-a815-bad89878e0d6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:15.946170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4476db2-d1e4-4175-9bc5-9962606fe5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:15.948391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4476db2-d1e4-4175-9bc5-9962606fe5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:15.948681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4476db2-d1e4-4175-9bc5-9962606fe5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:15.948997 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4476db2-d1e4-4175-9bc5-9962606fe5e9 admin admin] PATH_INFO: `/v3/projects/ca723eecd3294d5da38a641421d89680/users/c094b36674524f4396d3c323e34c877e/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:15.971794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4476db2-d1e4-4175-9bc5-9962606fe5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca723eecd3294d5da38a641421d89680, user_id=c094b36674524f4396d3c323e34c877e, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:15.976815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4476db2-d1e4-4175-9bc5-9962606fe5e9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:15.993823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 917/1814] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:15 2022] PUT /identity/v3/projects/ca723eecd3294d5da38a641421d89680/users/c094b36674524f4396d3c323e34c877e/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:16.006089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9ee65-ac71-4f66-8d26-bd237fcee013 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.006378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9ee65-ac71-4f66-8d26-bd237fcee013 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.006655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9ee65-ac71-4f66-8d26-bd237fcee013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.044439 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4ad9ee65-ac71-4f66-8d26-bd237fcee013 None None] MFA Rules not processed for user `c094b36674524f4396d3c323e34c877e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:16.046036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4ad9ee65-ac71-4f66-8d26-bd237fcee013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:16.136906 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 898/1815] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:16.144692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4476db2-d1e4-4175-9bc5-9962606fe5e9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.195092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:16.210331 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4476db2-d1e4-4175-9bc5-9962606fe5e9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:16.215073 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db0208-9b2f-43a1-9424-6240321fb81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:16.215676 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db0208-9b2f-43a1-9424-6240321fb81d admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.215955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db0208-9b2f-43a1-9424-6240321fb81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.216279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db0208-9b2f-43a1-9424-6240321fb81d admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.216946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db0208-9b2f-43a1-9424-6240321fb81d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:16.218870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db0208-9b2f-43a1-9424-6240321fb81d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:16.244947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 918/1816] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:16 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:16.249046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9ee65-ac71-4f66-8d26-bd237fcee013 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.261198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9ee65-ac71-4f66-8d26-bd237fcee013 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:16.285873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:16.347917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b1e19-fcfd-4b93-adea-a8540668929c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:16.348651 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b1e19-fcfd-4b93-adea-a8540668929c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.349180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b1e19-fcfd-4b93-adea-a8540668929c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.349474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b1e19-fcfd-4b93-adea-a8540668929c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.350122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b1e19-fcfd-4b93-adea-a8540668929c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:16.352368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b1e19-fcfd-4b93-adea-a8540668929c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:16.388763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 899/1817] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:16 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:16.394402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db0208-9b2f-43a1-9424-6240321fb81d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.404234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db0208-9b2f-43a1-9424-6240321fb81d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:16.410538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab78dc3-131b-41c2-b96e-9d60b0c58e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:16.411204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab78dc3-131b-41c2-b96e-9d60b0c58e91 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.411488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab78dc3-131b-41c2-b96e-9d60b0c58e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.411758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab78dc3-131b-41c2-b96e-9d60b0c58e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.412415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab78dc3-131b-41c2-b96e-9d60b0c58e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:16.414005 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab78dc3-131b-41c2-b96e-9d60b0c58e91 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:16.426940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 919/1818] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:16.430924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b1e19-fcfd-4b93-adea-a8540668929c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.447379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b1e19-fcfd-4b93-adea-a8540668929c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:16.455645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86892b69-e88a-4f11-a5dc-c05bf0f42510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:16.456545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86892b69-e88a-4f11-a5dc-c05bf0f42510 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.457041 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86892b69-e88a-4f11-a5dc-c05bf0f42510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.457396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86892b69-e88a-4f11-a5dc-c05bf0f42510 admin admin] PATH_INFO: `/v3/projects/c49d7ee8930f4a68919bd23aa2c21f1f/users/6ef4ec4b0e4047248d0132b2d0c8a29a/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.493597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86892b69-e88a-4f11-a5dc-c05bf0f42510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49d7ee8930f4a68919bd23aa2c21f1f, user_id=6ef4ec4b0e4047248d0132b2d0c8a29a, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:16.503689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86892b69-e88a-4f11-a5dc-c05bf0f42510 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:16.527092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 900/1819] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:16 2022] PUT /identity/v3/projects/c49d7ee8930f4a68919bd23aa2c21f1f/users/6ef4ec4b0e4047248d0132b2d0c8a29a/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:16.531415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab78dc3-131b-41c2-b96e-9d60b0c58e91 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.542348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab78dc3-131b-41c2-b96e-9d60b0c58e91 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:16.661133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afeab07-b7ff-4200-94d9-8715d50a5c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:16.665206 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afeab07-b7ff-4200-94d9-8715d50a5c84 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.665493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afeab07-b7ff-4200-94d9-8715d50a5c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.665768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afeab07-b7ff-4200-94d9-8715d50a5c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.668478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afeab07-b7ff-4200-94d9-8715d50a5c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:16.670128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afeab07-b7ff-4200-94d9-8715d50a5c84 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:16.705934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 920/1820] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:16.706759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86892b69-e88a-4f11-a5dc-c05bf0f42510 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.723649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86892b69-e88a-4f11-a5dc-c05bf0f42510 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:16.728671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995bbbd-3d23-4741-a520-faee649b33cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:16.729345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995bbbd-3d23-4741-a520-faee649b33cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.729629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995bbbd-3d23-4741-a520-faee649b33cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.729910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995bbbd-3d23-4741-a520-faee649b33cc admin admin] PATH_INFO: `/v3/projects/c49d7ee8930f4a68919bd23aa2c21f1f/users/6ef4ec4b0e4047248d0132b2d0c8a29a/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.732068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995bbbd-3d23-4741-a520-faee649b33cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49d7ee8930f4a68919bd23aa2c21f1f, user_id=6ef4ec4b0e4047248d0132b2d0c8a29a, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:16.735569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995bbbd-3d23-4741-a520-faee649b33cc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:16.752768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 901/1821] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:16 2022] PUT /identity/v3/projects/c49d7ee8930f4a68919bd23aa2c21f1f/users/6ef4ec4b0e4047248d0132b2d0c8a29a/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:16.763844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e854a2a-7395-4dde-b2c3-23304abb89fb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:16.764257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e854a2a-7395-4dde-b2c3-23304abb89fb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:16.764328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e854a2a-7395-4dde-b2c3-23304abb89fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:16.806287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8e854a2a-7395-4dde-b2c3-23304abb89fb None None] MFA Rules not processed for user `6ef4ec4b0e4047248d0132b2d0c8a29a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:16.808042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8e854a2a-7395-4dde-b2c3-23304abb89fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:16.917291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 921/1822] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:16.932126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995bbbd-3d23-4741-a520-faee649b33cc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:16.933864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b995bbbd-3d23-4741-a520-faee649b33cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:17.040577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995bbbd-3d23-4741-a520-faee649b33cc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:17.044963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62060930-87a9-4c55-ac28-eef1585e3690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:17.045738 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060930-87a9-4c55-ac28-eef1585e3690 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:17.045870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060930-87a9-4c55-ac28-eef1585e3690 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:17.046765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060930-87a9-4c55-ac28-eef1585e3690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:17.056889 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62060930-87a9-4c55-ac28-eef1585e3690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:17.058382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62060930-87a9-4c55-ac28-eef1585e3690 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:17.069987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 902/1823] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:02:17.120084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e854a2a-7395-4dde-b2c3-23304abb89fb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:17.262876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:17.307777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e854a2a-7395-4dde-b2c3-23304abb89fb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:17.323516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:17.323516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:17.323516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:17.323516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] PATH_INFO: `/v3/users/c094b36674524f4396d3c323e34c877e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:17.326542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c094b36674524f4396d3c323e34c877e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:17.328590 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:17.459323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c094b36674524f4396d3c323e34c877e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:17.460431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c094b36674524f4396d3c323e34c877e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:17.479826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c094b36674524f4396d3c323e34c877e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:17.491818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c094b36674524f4396d3c323e34c877e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:17.505103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c094b36674524f4396d3c323e34c877e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:17.511709 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 922/1824] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:17 2022] DELETE /identity/v3/users/c094b36674524f4396d3c323e34c877e => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:17.516402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db60a1-c5a0-47e8-9cd6-b4492ddc8a44 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:17.516928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db60a1-c5a0-47e8-9cd6-b4492ddc8a44 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:17.516928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db60a1-c5a0-47e8-9cd6-b4492ddc8a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:17.541264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-24db60a1-c5a0-47e8-9cd6-b4492ddc8a44 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:17.542749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-24db60a1-c5a0-47e8-9cd6-b4492ddc8a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:17.644425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 903/1825] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:17.654575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:17.656688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:17.739216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db60a1-c5a0-47e8-9cd6-b4492ddc8a44 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:17.740959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-24db60a1-c5a0-47e8-9cd6-b4492ddc8a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:17.756286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db60a1-c5a0-47e8-9cd6-b4492ddc8a44 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:17.765301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:17.765301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:17.765301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:17.765788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] PATH_INFO: `/v3/users/33974ab4030e4125b89c593fca3e5175` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:17.767008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] RBAC: Authorizing `identity:delete_user(user_id=33974ab4030e4125b89c593fca3e5175)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:17.769535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:17.787922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:17.805500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63645e85-be09-47e8-87bc-8f7f40eba518 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:17.819436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b936f58-3725-4083-9742-85e88cc9018c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:17.820224 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b936f58-3725-4083-9742-85e88cc9018c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:17.820660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b936f58-3725-4083-9742-85e88cc9018c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:17.821015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b936f58-3725-4083-9742-85e88cc9018c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:17.832592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b936f58-3725-4083-9742-85e88cc9018c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:17.834328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b936f58-3725-4083-9742-85e88cc9018c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:17.847591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 923/1826] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:17.852549 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33974ab4030e4125b89c593fca3e5175'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:17.868814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33974ab4030e4125b89c593fca3e5175'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:17.869758 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33974ab4030e4125b89c593fca3e5175'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:17.870718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33974ab4030e4125b89c593fca3e5175'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:17.888113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33974ab4030e4125b89c593fca3e5175'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:17.904151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 904/1827] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:17 2022] DELETE /identity/v3/users/33974ab4030e4125b89c593fca3e5175 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:17.909936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71af36-05f5-4738-bbe3-4b526811429c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:17.910085 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71af36-05f5-4738-bbe3-4b526811429c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:17.910395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71af36-05f5-4738-bbe3-4b526811429c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:17.932602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bd71af36-05f5-4738-bbe3-4b526811429c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:17.933959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bd71af36-05f5-4738-bbe3-4b526811429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:18.023700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 924/1828] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:18.036094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:18.037494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:18.108098 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d2ae1-c8ba-4f9f-b77b-2fe0c6f4a5af admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:18.112087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986a143-43d3-4449-985f-dad87cf8989a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:18.112703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986a143-43d3-4449-985f-dad87cf8989a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:18.112985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986a143-43d3-4449-985f-dad87cf8989a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:18.113305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986a143-43d3-4449-985f-dad87cf8989a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:18.123582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986a143-43d3-4449-985f-dad87cf8989a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:18.125105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986a143-43d3-4449-985f-dad87cf8989a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:18.135513 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 905/1829] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:18.192337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71af36-05f5-4738-bbe3-4b526811429c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:18.194082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bd71af36-05f5-4738-bbe3-4b526811429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:18.207244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71af36-05f5-4738-bbe3-4b526811429c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:18.213977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:18.215334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:18.215801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:18.216093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] PATH_INFO: `/v3/projects/ca723eecd3294d5da38a641421d89680` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:18.217397 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca723eecd3294d5da38a641421d89680)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:18.220120 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:18.222396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:18.279146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca723eecd3294d5da38a641421d89680', 'request_id': 'req-e176f685-6635-4112-a471-57402fc7bca7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.287965 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca723eecd3294d5da38a641421d89680', 'request_id': 'req-e176f685-6635-4112-a471-57402fc7bca7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.295742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca723eecd3294d5da38a641421d89680', 'request_id': 'req-e176f685-6635-4112-a471-57402fc7bca7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.307179 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] The token cache is being invalidate because project ca723eecd3294d5da38a641421d89680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:18.307668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.308704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.310263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 925/1830] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:18 2022] DELETE /identity/v3/projects/ca723eecd3294d5da38a641421d89680 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:18.315065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986a143-43d3-4449-985f-dad87cf8989a service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:18.325277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986a143-43d3-4449-985f-dad87cf8989a service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:18.399241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:18.401352 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:18.401352 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:18.401352 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] PATH_INFO: `/v3/users/6ef4ec4b0e4047248d0132b2d0c8a29a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:18.402440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef4ec4b0e4047248d0132b2d0c8a29a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:18.404511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:18.449995 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:18.458295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:18.460434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:18.467535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef4ec4b0e4047248d0132b2d0c8a29a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.479991 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176f685-6635-4112-a471-57402fc7bca7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:18.485884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef4ec4b0e4047248d0132b2d0c8a29a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.487742 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef4ec4b0e4047248d0132b2d0c8a29a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.488609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef4ec4b0e4047248d0132b2d0c8a29a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.492289 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:18.493071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:18.493211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:18.493449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] PATH_INFO: `/v3/projects/d51c4735cda74af58b39986751b12313` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:18.495796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d51c4735cda74af58b39986751b12313)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:18.498481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:18.501640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef4ec4b0e4047248d0132b2d0c8a29a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.518048 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 906/1831] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:18 2022] DELETE /identity/v3/users/6ef4ec4b0e4047248d0132b2d0c8a29a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:18.562369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd51c4735cda74af58b39986751b12313', 'request_id': 'req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.572334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd51c4735cda74af58b39986751b12313', 'request_id': 'req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.582226 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd51c4735cda74af58b39986751b12313', 'request_id': 'req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.595697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] The token cache is being invalidate because project d51c4735cda74af58b39986751b12313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:18.595876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.596574 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.598366 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 926/1832] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:18 2022] DELETE /identity/v3/projects/d51c4735cda74af58b39986751b12313 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:18.603434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:18.605080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:18.698898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82afa3-636e-4cbb-8b8a-5eb5958a7a48 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:18.780013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fc40e-623f-411f-bc13-306191bbf5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:18.780901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc40e-623f-411f-bc13-306191bbf5d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:18.781258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc40e-623f-411f-bc13-306191bbf5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:18.786609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc40e-623f-411f-bc13-306191bbf5d9 admin admin] PATH_INFO: `/v3/users/2208d58795ff44b4b5ca08d7397b5983` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:18.786609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fc40e-623f-411f-bc13-306191bbf5d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2208d58795ff44b4b5ca08d7397b5983)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:18.786609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fc40e-623f-411f-bc13-306191bbf5d9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:18.853137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-100fc40e-623f-411f-bc13-306191bbf5d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2208d58795ff44b4b5ca08d7397b5983'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.854568 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:18.856177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:18.865702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:18.868415 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-100fc40e-623f-411f-bc13-306191bbf5d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2208d58795ff44b4b5ca08d7397b5983'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.869512 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-100fc40e-623f-411f-bc13-306191bbf5d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2208d58795ff44b4b5ca08d7397b5983'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.870576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-100fc40e-623f-411f-bc13-306191bbf5d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2208d58795ff44b4b5ca08d7397b5983'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.877804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00803e64-e4ca-45ce-9f51-e4a3a66f5c23 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:18.897893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-100fc40e-623f-411f-bc13-306191bbf5d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2208d58795ff44b4b5ca08d7397b5983'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:18.899558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:18.901417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:18.901714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:18.901992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] PATH_INFO: `/v3/projects/c49d7ee8930f4a68919bd23aa2c21f1f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:18.905486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c49d7ee8930f4a68919bd23aa2c21f1f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:18.909311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:18.945511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 907/1833] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:18 2022] DELETE /identity/v3/users/2208d58795ff44b4b5ca08d7397b5983 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:19.006788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c49d7ee8930f4a68919bd23aa2c21f1f', 'request_id': 'req-3e10ba42-671e-4885-af1a-35fb06ad0868'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:19.017509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c49d7ee8930f4a68919bd23aa2c21f1f', 'request_id': 'req-3e10ba42-671e-4885-af1a-35fb06ad0868'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:19.028125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c49d7ee8930f4a68919bd23aa2c21f1f', 'request_id': 'req-3e10ba42-671e-4885-af1a-35fb06ad0868'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:19.036644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=76,keystone:DELETE=21,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:19.047958 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] The token cache is being invalidate because project c49d7ee8930f4a68919bd23aa2c21f1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:19.048523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:19.050067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3e10ba42-671e-4885-af1a-35fb06ad0868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:19.055211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 927/1834] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:18 2022] DELETE /identity/v3/projects/c49d7ee8930f4a68919bd23aa2c21f1f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:19.058586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c63458-8f3d-4d7f-b19a-db10ff532df0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.058926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c63458-8f3d-4d7f-b19a-db10ff532df0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.059231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c63458-8f3d-4d7f-b19a-db10ff532df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.085849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-35c63458-8f3d-4d7f-b19a-db10ff532df0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:19.087648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-35c63458-8f3d-4d7f-b19a-db10ff532df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:19.189531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 908/1835] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:19.192455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c761379-374d-48f2-8024-83f36995746b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.193002 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c761379-374d-48f2-8024-83f36995746b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.193275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c761379-374d-48f2-8024-83f36995746b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.215028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9c761379-374d-48f2-8024-83f36995746b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:19.216238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9c761379-374d-48f2-8024-83f36995746b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:19.225005 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 928/1836] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:19.229253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c63458-8f3d-4d7f-b19a-db10ff532df0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.240191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c63458-8f3d-4d7f-b19a-db10ff532df0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.244878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75c7d9-8a81-49a4-8caa-44f58200acf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.245486 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75c7d9-8a81-49a4-8caa-44f58200acf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.245768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75c7d9-8a81-49a4-8caa-44f58200acf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.246042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75c7d9-8a81-49a4-8caa-44f58200acf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.246688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75c7d9-8a81-49a4-8caa-44f58200acf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:19.246990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75c7d9-8a81-49a4-8caa-44f58200acf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.248618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75c7d9-8a81-49a4-8caa-44f58200acf6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.260616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 909/1837] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:19.265140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c761379-374d-48f2-8024-83f36995746b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.274527 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c761379-374d-48f2-8024-83f36995746b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.279930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccb482-ab29-47eb-8fce-5b2b09ab78cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.281105 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb482-ab29-47eb-8fce-5b2b09ab78cb admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.281384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb482-ab29-47eb-8fce-5b2b09ab78cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.281653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb482-ab29-47eb-8fce-5b2b09ab78cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.282397 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccb482-ab29-47eb-8fce-5b2b09ab78cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.284234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccb482-ab29-47eb-8fce-5b2b09ab78cb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.318335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 929/1838] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:19.322761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75c7d9-8a81-49a4-8caa-44f58200acf6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.334851 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75c7d9-8a81-49a4-8caa-44f58200acf6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.385458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccb482-ab29-47eb-8fce-5b2b09ab78cb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.387280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0fccb482-ab29-47eb-8fce-5b2b09ab78cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:19.402809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a13a59-1a90-4998-8e62-66378cb3de2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.403515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13a59-1a90-4998-8e62-66378cb3de2c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.403744 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13a59-1a90-4998-8e62-66378cb3de2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.404010 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13a59-1a90-4998-8e62-66378cb3de2c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.404640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a13a59-1a90-4998-8e62-66378cb3de2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.407157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a13a59-1a90-4998-8e62-66378cb3de2c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.431543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 910/1839] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:19.435499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a13a59-1a90-4998-8e62-66378cb3de2c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.446851 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a13a59-1a90-4998-8e62-66378cb3de2c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.451633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c06a17-eea2-448e-9dde-74c19f490c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.452829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c06a17-eea2-448e-9dde-74c19f490c7b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.453114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c06a17-eea2-448e-9dde-74c19f490c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.453389 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c06a17-eea2-448e-9dde-74c19f490c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.454025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c06a17-eea2-448e-9dde-74c19f490c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.455677 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c06a17-eea2-448e-9dde-74c19f490c7b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.463356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccb482-ab29-47eb-8fce-5b2b09ab78cb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.466987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 911/1840] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:19.468656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.469323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.469654 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.470046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] PATH_INFO: `/v3/projects/35bd38302d9d41bfbb9b9400ceeb29a6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.471644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c06a17-eea2-448e-9dde-74c19f490c7b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.474838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35bd38302d9d41bfbb9b9400ceeb29a6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.477743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.482440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c06a17-eea2-448e-9dde-74c19f490c7b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.489093 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a7eb9-ad40-449a-a9ce-ebba9b36bda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.489698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a7eb9-ad40-449a-a9ce-ebba9b36bda4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.489980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a7eb9-ad40-449a-a9ce-ebba9b36bda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.490254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a7eb9-ad40-449a-a9ce-ebba9b36bda4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.490886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a7eb9-ad40-449a-a9ce-ebba9b36bda4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.492473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a7eb9-ad40-449a-a9ce-ebba9b36bda4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.506873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 912/1841] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:19.507879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a7eb9-ad40-449a-a9ce-ebba9b36bda4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.518284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a7eb9-ad40-449a-a9ce-ebba9b36bda4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.529467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35bd38302d9d41bfbb9b9400ceeb29a6', 'request_id': 'req-b9b184aa-8ac2-42e1-9ee8-955d5e854643'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:19.537097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35bd38302d9d41bfbb9b9400ceeb29a6', 'request_id': 'req-b9b184aa-8ac2-42e1-9ee8-955d5e854643'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:19.543685 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35bd38302d9d41bfbb9b9400ceeb29a6', 'request_id': 'req-b9b184aa-8ac2-42e1-9ee8-955d5e854643'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:19.549409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:19.555121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] The token cache is being invalidate because project 35bd38302d9d41bfbb9b9400ceeb29a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:19.555416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:19.556093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b9b184aa-8ac2-42e1-9ee8-955d5e854643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:19.557594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 930/1842] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:19 2022] DELETE /identity/v3/projects/35bd38302d9d41bfbb9b9400ceeb29a6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:19.563316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f3208-76e6-44d1-b891-144c9ee8011b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.563596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f3208-76e6-44d1-b891-144c9ee8011b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.563876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f3208-76e6-44d1-b891-144c9ee8011b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.580114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3cd2b-4284-4fcf-a565-19878a38c9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.580772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3cd2b-4284-4fcf-a565-19878a38c9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.581051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3cd2b-4284-4fcf-a565-19878a38c9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.581435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3cd2b-4284-4fcf-a565-19878a38c9b2 admin admin] PATH_INFO: `/v3/projects/9693a11e6b5a42acbe830515f584484e/users/b97684f6fd344708a0634f467a561d02/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.585182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ff1f3208-76e6-44d1-b891-144c9ee8011b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:19.586385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ff1f3208-76e6-44d1-b891-144c9ee8011b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:19.594512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 931/1843] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:19.595703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3cd2b-4284-4fcf-a565-19878a38c9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9693a11e6b5a42acbe830515f584484e, user_id=b97684f6fd344708a0634f467a561d02, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.598658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3cd2b-4284-4fcf-a565-19878a38c9b2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.602373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54353c-2fdb-43b3-846b-d6b8660fa198 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.602695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54353c-2fdb-43b3-846b-d6b8660fa198 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.603022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54353c-2fdb-43b3-846b-d6b8660fa198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.619082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 913/1844] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:19 2022] PUT /identity/v3/projects/9693a11e6b5a42acbe830515f584484e/users/b97684f6fd344708a0634f467a561d02/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:19.623442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ab54353c-2fdb-43b3-846b-d6b8660fa198 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:19.625072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1bee6-32fe-4071-b7ff-0e43f55ddd32 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.625332 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1bee6-32fe-4071-b7ff-0e43f55ddd32 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.625466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ab54353c-2fdb-43b3-846b-d6b8660fa198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:19.625748 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1bee6-32fe-4071-b7ff-0e43f55ddd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.653334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bec1bee6-32fe-4071-b7ff-0e43f55ddd32 None None] MFA Rules not processed for user `b97684f6fd344708a0634f467a561d02`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:19.654527 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bec1bee6-32fe-4071-b7ff-0e43f55ddd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:19.669617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:19.690852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 932/1845] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:19.695210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54353c-2fdb-43b3-846b-d6b8660fa198 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.707462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54353c-2fdb-43b3-846b-d6b8660fa198 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.713852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a306a-59fb-4edb-9cfa-1af97c246cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.714810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a306a-59fb-4edb-9cfa-1af97c246cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.714810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a306a-59fb-4edb-9cfa-1af97c246cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.715018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a306a-59fb-4edb-9cfa-1af97c246cc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.715594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a306a-59fb-4edb-9cfa-1af97c246cc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:19.715898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a306a-59fb-4edb-9cfa-1af97c246cc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.717462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a306a-59fb-4edb-9cfa-1af97c246cc2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.727719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 914/1846] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:19.732196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1bee6-32fe-4071-b7ff-0e43f55ddd32 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.734003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bec1bee6-32fe-4071-b7ff-0e43f55ddd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:19.737527 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 933/1847] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:02:19.738937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a306a-59fb-4edb-9cfa-1af97c246cc2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.747379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1bee6-32fe-4071-b7ff-0e43f55ddd32 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.748587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a306a-59fb-4edb-9cfa-1af97c246cc2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.752618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deea7d6-6366-4a2c-8690-78063ea3ca5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.753242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deea7d6-6366-4a2c-8690-78063ea3ca5b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.753521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deea7d6-6366-4a2c-8690-78063ea3ca5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.753792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deea7d6-6366-4a2c-8690-78063ea3ca5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.753895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be012c51-37fd-4c7c-b240-ecb50e3e7075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.754446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deea7d6-6366-4a2c-8690-78063ea3ca5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.754541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be012c51-37fd-4c7c-b240-ecb50e3e7075 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.754726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be012c51-37fd-4c7c-b240-ecb50e3e7075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.755021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be012c51-37fd-4c7c-b240-ecb50e3e7075 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.755640 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be012c51-37fd-4c7c-b240-ecb50e3e7075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.756357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deea7d6-6366-4a2c-8690-78063ea3ca5b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.757504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be012c51-37fd-4c7c-b240-ecb50e3e7075 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.784615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 915/1848] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:19.785872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 934/1849] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:19.788807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deea7d6-6366-4a2c-8690-78063ea3ca5b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.789726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be012c51-37fd-4c7c-b240-ecb50e3e7075 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.850368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deea7d6-6366-4a2c-8690-78063ea3ca5b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.854975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f70fd5-f29d-485f-9f59-d870c56e2d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.855475 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be012c51-37fd-4c7c-b240-ecb50e3e7075 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.855931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f70fd5-f29d-485f-9f59-d870c56e2d4e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.855931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f70fd5-f29d-485f-9f59-d870c56e2d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.856105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f70fd5-f29d-485f-9f59-d870c56e2d4e admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.856746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f70fd5-f29d-485f-9f59-d870c56e2d4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.859202 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f70fd5-f29d-485f-9f59-d870c56e2d4e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.861169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086522e3-4d1b-4957-9c91-c06165f598ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.861768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086522e3-4d1b-4957-9c91-c06165f598ad admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.862092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086522e3-4d1b-4957-9c91-c06165f598ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.862539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086522e3-4d1b-4957-9c91-c06165f598ad admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.863262 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086522e3-4d1b-4957-9c91-c06165f598ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.865149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086522e3-4d1b-4957-9c91-c06165f598ad admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.898336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 916/1850] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:19.898336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f70fd5-f29d-485f-9f59-d870c56e2d4e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.898733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 935/1851] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:19 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:19.898733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086522e3-4d1b-4957-9c91-c06165f598ad admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.914222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f70fd5-f29d-485f-9f59-d870c56e2d4e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.919161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086522e3-4d1b-4957-9c91-c06165f598ad admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.928877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8560a8b-acd7-4527-8bf9-605c4629c763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.929874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8560a8b-acd7-4527-8bf9-605c4629c763 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.930157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8560a8b-acd7-4527-8bf9-605c4629c763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.930431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8560a8b-acd7-4527-8bf9-605c4629c763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.931250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8560a8b-acd7-4527-8bf9-605c4629c763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.931532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b0d91-5477-4277-9396-40f24add7eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.932153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b0d91-5477-4277-9396-40f24add7eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.932428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b0d91-5477-4277-9396-40f24add7eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.932864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b0d91-5477-4277-9396-40f24add7eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.932959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8560a8b-acd7-4527-8bf9-605c4629c763 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.933511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b0d91-5477-4277-9396-40f24add7eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.935118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b0d91-5477-4277-9396-40f24add7eb6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.944180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 917/1852] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:19.947840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 936/1853] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:19.948725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b0d91-5477-4277-9396-40f24add7eb6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.950701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8560a8b-acd7-4527-8bf9-605c4629c763 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.955981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:19.959302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b0d91-5477-4277-9396-40f24add7eb6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.961319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8560a8b-acd7-4527-8bf9-605c4629c763 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:19.964511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77077cc1-cb0e-43f6-9fe2-096cbc4a3d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.965196 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77077cc1-cb0e-43f6-9fe2-096cbc4a3d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.965482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77077cc1-cb0e-43f6-9fe2-096cbc4a3d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.965757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77077cc1-cb0e-43f6-9fe2-096cbc4a3d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.966437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77077cc1-cb0e-43f6-9fe2-096cbc4a3d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.966952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af08277-17e0-45dd-b4fd-20343d816e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:19.967585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08277-17e0-45dd-b4fd-20343d816e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:19.968353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08277-17e0-45dd-b4fd-20343d816e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:19.968635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08277-17e0-45dd-b4fd-20343d816e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:19.968766 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77077cc1-cb0e-43f6-9fe2-096cbc4a3d91 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.969267 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af08277-17e0-45dd-b4fd-20343d816e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:19.970838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af08277-17e0-45dd-b4fd-20343d816e5a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:19.983593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 937/1854] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:19.983725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 918/1855] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:19.984487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77077cc1-cb0e-43f6-9fe2-096cbc4a3d91 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.986382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af08277-17e0-45dd-b4fd-20343d816e5a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:19.997878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af08277-17e0-45dd-b4fd-20343d816e5a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.002419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77077cc1-cb0e-43f6-9fe2-096cbc4a3d91 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.003300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dcc91b-0b0b-4a79-98f6-5b3d67d4d630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.004138 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcc91b-0b0b-4a79-98f6-5b3d67d4d630 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.004567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcc91b-0b0b-4a79-98f6-5b3d67d4d630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.004997 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcc91b-0b0b-4a79-98f6-5b3d67d4d630 admin admin] PATH_INFO: `/v3/projects/3c8ee98b32ef4d18b37fda8dc8bf9b63/users/4e4179e482d94a3eb707491c442cfd7c/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.008738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364bf1d-c004-4036-8eee-5a87f0a142e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.009662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364bf1d-c004-4036-8eee-5a87f0a142e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.010096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364bf1d-c004-4036-8eee-5a87f0a142e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.010467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364bf1d-c004-4036-8eee-5a87f0a142e6 admin admin] PATH_INFO: `/v3/projects/73ca9ab5e4624ea3aabba4d5df2300e2/users/b5a8883367d54071bedb46733891c03a/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.032838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364bf1d-c004-4036-8eee-5a87f0a142e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ca9ab5e4624ea3aabba4d5df2300e2, user_id=b5a8883367d54071bedb46733891c03a, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.036064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364bf1d-c004-4036-8eee-5a87f0a142e6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.039866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dcc91b-0b0b-4a79-98f6-5b3d67d4d630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c8ee98b32ef4d18b37fda8dc8bf9b63, user_id=4e4179e482d94a3eb707491c442cfd7c, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.043541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dcc91b-0b0b-4a79-98f6-5b3d67d4d630 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.061072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 938/1856] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:19 2022] PUT /identity/v3/projects/73ca9ab5e4624ea3aabba4d5df2300e2/users/b5a8883367d54071bedb46733891c03a/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:20.073834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 919/1857] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:19 2022] PUT /identity/v3/projects/3c8ee98b32ef4d18b37fda8dc8bf9b63/users/4e4179e482d94a3eb707491c442cfd7c/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:20.074845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb599bfe-ab24-411e-a7f9-6acef2f6003b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.075172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb599bfe-ab24-411e-a7f9-6acef2f6003b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.075377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148a76-1f36-4090-b57d-835941359fd8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.075460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb599bfe-ab24-411e-a7f9-6acef2f6003b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.075648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148a76-1f36-4090-b57d-835941359fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.075932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148a76-1f36-4090-b57d-835941359fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.123495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9f148a76-1f36-4090-b57d-835941359fd8 None None] MFA Rules not processed for user `4e4179e482d94a3eb707491c442cfd7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:20.124113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-eb599bfe-ab24-411e-a7f9-6acef2f6003b None None] MFA Rules not processed for user `b5a8883367d54071bedb46733891c03a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:20.125439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eb599bfe-ab24-411e-a7f9-6acef2f6003b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:20.125601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9f148a76-1f36-4090-b57d-835941359fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:20.210067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:20.241512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 939/1858] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:20.242532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb599bfe-ab24-411e-a7f9-6acef2f6003b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.249251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 920/1859] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:20.258530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f148a76-1f36-4090-b57d-835941359fd8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.260752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9f148a76-1f36-4090-b57d-835941359fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:20.338848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb599bfe-ab24-411e-a7f9-6acef2f6003b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.344174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd619f-c7d8-431f-9fc3-41518542fd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.345146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd619f-c7d8-431f-9fc3-41518542fd72 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.345146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd619f-c7d8-431f-9fc3-41518542fd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.345369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd619f-c7d8-431f-9fc3-41518542fd72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.345972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd619f-c7d8-431f-9fc3-41518542fd72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.347908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd619f-c7d8-431f-9fc3-41518542fd72 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.363679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f148a76-1f36-4090-b57d-835941359fd8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.373888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cad10-4e72-4c00-b47e-a6dc48072719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.374575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cad10-4e72-4c00-b47e-a6dc48072719 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.374854 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cad10-4e72-4c00-b47e-a6dc48072719 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.375169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cad10-4e72-4c00-b47e-a6dc48072719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.394260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615cad10-4e72-4c00-b47e-a6dc48072719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.395831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615cad10-4e72-4c00-b47e-a6dc48072719 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.402666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 940/1860] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:20 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:20.403888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd619f-c7d8-431f-9fc3-41518542fd72 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.405951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-95cd619f-c7d8-431f-9fc3-41518542fd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:20.414421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 921/1861] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:20.415208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cad10-4e72-4c00-b47e-a6dc48072719 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.428053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cad10-4e72-4c00-b47e-a6dc48072719 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.485096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd619f-c7d8-431f-9fc3-41518542fd72 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.491456 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d493b-b820-4d59-99a9-47def9c13a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.492216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d493b-b820-4d59-99a9-47def9c13a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.492696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d493b-b820-4d59-99a9-47def9c13a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.492974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d493b-b820-4d59-99a9-47def9c13a5b admin admin] PATH_INFO: `/v3/users/25a6f28de3e54d7ca1cd44cd9704fb04` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.494580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d493b-b820-4d59-99a9-47def9c13a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=25a6f28de3e54d7ca1cd44cd9704fb04)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.496892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d493b-b820-4d59-99a9-47def9c13a5b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.511611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a010e7-187d-45c4-9fe5-80688629eb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.513035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a010e7-187d-45c4-9fe5-80688629eb6b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.513413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a010e7-187d-45c4-9fe5-80688629eb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.513784 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a010e7-187d-45c4-9fe5-80688629eb6b admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.514540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a010e7-187d-45c4-9fe5-80688629eb6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.517245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a010e7-187d-45c4-9fe5-80688629eb6b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.527556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:20.553060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 922/1862] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:20 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:20.557694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a010e7-187d-45c4-9fe5-80688629eb6b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.570060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a010e7-187d-45c4-9fe5-80688629eb6b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.578339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066da853-ea1f-4995-bc95-40223cab1ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.579227 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066da853-ea1f-4995-bc95-40223cab1ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.579527 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a59d493b-b820-4d59-99a9-47def9c13a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25a6f28de3e54d7ca1cd44cd9704fb04'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:20.579665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066da853-ea1f-4995-bc95-40223cab1ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.579983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066da853-ea1f-4995-bc95-40223cab1ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.580542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a59d493b-b820-4d59-99a9-47def9c13a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25a6f28de3e54d7ca1cd44cd9704fb04'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:20.580832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066da853-ea1f-4995-bc95-40223cab1ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.583302 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066da853-ea1f-4995-bc95-40223cab1ee8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.597586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 923/1863] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:20.597813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a59d493b-b820-4d59-99a9-47def9c13a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25a6f28de3e54d7ca1cd44cd9704fb04'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:20.599162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066da853-ea1f-4995-bc95-40223cab1ee8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.601220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-066da853-ea1f-4995-bc95-40223cab1ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:20.605465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a59d493b-b820-4d59-99a9-47def9c13a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25a6f28de3e54d7ca1cd44cd9704fb04'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:20.613566 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:20.615015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a59d493b-b820-4d59-99a9-47def9c13a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25a6f28de3e54d7ca1cd44cd9704fb04'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:20.622490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 941/1864] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:20 2022] DELETE /identity/v3/users/25a6f28de3e54d7ca1cd44cd9704fb04 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:20.633350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066da853-ea1f-4995-bc95-40223cab1ee8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.637135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7f4e0-ca24-47cc-aecd-6ee07aa09883 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.637427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7f4e0-ca24-47cc-aecd-6ee07aa09883 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.637922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7f4e0-ca24-47cc-aecd-6ee07aa09883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.661989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6af7f4e0-ca24-47cc-aecd-6ee07aa09883 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:20.663247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6af7f4e0-ca24-47cc-aecd-6ee07aa09883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:20.705594 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67529aa8-3622-4082-a7b2-2ae8b978ae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.706238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67529aa8-3622-4082-a7b2-2ae8b978ae14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.707237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67529aa8-3622-4082-a7b2-2ae8b978ae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.707237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67529aa8-3622-4082-a7b2-2ae8b978ae14 admin admin] PATH_INFO: `/v3/projects/4c65cc0ed5e14982942f9ecb3aab8cf9/users/5ac174cfb0c549bab4c327d0875e99a3/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.731529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67529aa8-3622-4082-a7b2-2ae8b978ae14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c65cc0ed5e14982942f9ecb3aab8cf9, user_id=5ac174cfb0c549bab4c327d0875e99a3, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.731529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67529aa8-3622-4082-a7b2-2ae8b978ae14 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.740412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 942/1865] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:20.752274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7f4e0-ca24-47cc-aecd-6ee07aa09883 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.756576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6af7f4e0-ca24-47cc-aecd-6ee07aa09883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:20.761394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 924/1866] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:20 2022] PUT /identity/v3/projects/4c65cc0ed5e14982942f9ecb3aab8cf9/users/5ac174cfb0c549bab4c327d0875e99a3/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:20.766602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67529aa8-3622-4082-a7b2-2ae8b978ae14 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.825773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7f4e0-ca24-47cc-aecd-6ee07aa09883 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.845986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67529aa8-3622-4082-a7b2-2ae8b978ae14 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.850936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1190e-e859-4b45-a008-01889919938a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.851664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1190e-e859-4b45-a008-01889919938a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.851945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1190e-e859-4b45-a008-01889919938a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.852301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1190e-e859-4b45-a008-01889919938a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.852945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1190e-e859-4b45-a008-01889919938a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.854524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1190e-e859-4b45-a008-01889919938a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.865094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 925/1867] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:20.869307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1190e-e859-4b45-a008-01889919938a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:20.878940 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1190e-e859-4b45-a008-01889919938a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:20.880182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449ac43-84c8-4750-bba7-07a79166910e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.880796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449ac43-84c8-4750-bba7-07a79166910e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.881069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449ac43-84c8-4750-bba7-07a79166910e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.881333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449ac43-84c8-4750-bba7-07a79166910e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.883550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880c871-7c6a-40bb-985f-c8fb9726382a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:20.884180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880c871-7c6a-40bb-985f-c8fb9726382a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.884461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880c871-7c6a-40bb-985f-c8fb9726382a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.884732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880c871-7c6a-40bb-985f-c8fb9726382a admin admin] PATH_INFO: `/v3/projects/4c65cc0ed5e14982942f9ecb3aab8cf9/users/5ac174cfb0c549bab4c327d0875e99a3/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.886612 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880c871-7c6a-40bb-985f-c8fb9726382a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c65cc0ed5e14982942f9ecb3aab8cf9, user_id=5ac174cfb0c549bab4c327d0875e99a3, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.889265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880c871-7c6a-40bb-985f-c8fb9726382a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.890933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449ac43-84c8-4750-bba7-07a79166910e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:20.892401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449ac43-84c8-4750-bba7-07a79166910e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:20.901199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 926/1868] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:20 2022] PUT /identity/v3/projects/4c65cc0ed5e14982942f9ecb3aab8cf9/users/5ac174cfb0c549bab4c327d0875e99a3/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:20.902255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 943/1869] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:20.914618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30232179-85dd-42f9-952e-58b463c96942 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:20.914912 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30232179-85dd-42f9-952e-58b463c96942 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:20.915219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30232179-85dd-42f9-952e-58b463c96942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:20.944103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-30232179-85dd-42f9-952e-58b463c96942 None None] MFA Rules not processed for user `5ac174cfb0c549bab4c327d0875e99a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:20.945334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-30232179-85dd-42f9-952e-58b463c96942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:20.981633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:21.021294 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 927/1870] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:21.032774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449ac43-84c8-4750-bba7-07a79166910e service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:21.034281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d449ac43-84c8-4750-bba7-07a79166910e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:21.105507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449ac43-84c8-4750-bba7-07a79166910e service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:21.110103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:21.161999 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d45ab-5d2b-4f52-bf0c-6bf3899e3259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:21.162669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d45ab-5d2b-4f52-bf0c-6bf3899e3259 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:21.162968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d45ab-5d2b-4f52-bf0c-6bf3899e3259 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:21.163302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d45ab-5d2b-4f52-bf0c-6bf3899e3259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:21.172925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d45ab-5d2b-4f52-bf0c-6bf3899e3259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:21.174480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d45ab-5d2b-4f52-bf0c-6bf3899e3259 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:21.182070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30232179-85dd-42f9-952e-58b463c96942 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:21.183785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-30232179-85dd-42f9-952e-58b463c96942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:21.184924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 944/1871] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:21.253942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30232179-85dd-42f9-952e-58b463c96942 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:21.258465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:21.259132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:21.259410 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:21.259680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] PATH_INFO: `/v3/projects/ec2fbf1e6d1140de858b2e1a3f52b71e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:21.260788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2fbf1e6d1140de858b2e1a3f52b71e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:21.262667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:21.328876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2fbf1e6d1140de858b2e1a3f52b71e', 'request_id': 'req-d86ba05b-a635-4e28-80cb-5de8037e26ac'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:21.337208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2fbf1e6d1140de858b2e1a3f52b71e', 'request_id': 'req-d86ba05b-a635-4e28-80cb-5de8037e26ac'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:21.358615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2fbf1e6d1140de858b2e1a3f52b71e', 'request_id': 'req-d86ba05b-a635-4e28-80cb-5de8037e26ac'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:21.379600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] The token cache is being invalidate because project ec2fbf1e6d1140de858b2e1a3f52b71e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:21.379600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:21.379600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:21.380550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 928/1872] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:21 2022] DELETE /identity/v3/projects/ec2fbf1e6d1140de858b2e1a3f52b71e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:21.384403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d45ab-5d2b-4f52-bf0c-6bf3899e3259 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:21.398293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d45ab-5d2b-4f52-bf0c-6bf3899e3259 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:21.468711 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:21.469373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:21.469656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:21.470375 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] PATH_INFO: `/v3/users/a24149ab7ddd4ec08ac9c0b3c0b9cefd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:21.471278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a24149ab7ddd4ec08ac9c0b3c0b9cefd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:21.473206 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:21.538369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a24149ab7ddd4ec08ac9c0b3c0b9cefd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:21.539059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a24149ab7ddd4ec08ac9c0b3c0b9cefd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:21.556793 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a24149ab7ddd4ec08ac9c0b3c0b9cefd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:21.566339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a24149ab7ddd4ec08ac9c0b3c0b9cefd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:21.576263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a24149ab7ddd4ec08ac9c0b3c0b9cefd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:21.585007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 945/1873] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:21 2022] DELETE /identity/v3/users/a24149ab7ddd4ec08ac9c0b3c0b9cefd => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:21.832224 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:21.833646 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:21.914323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ba05b-a635-4e28-80cb-5de8037e26ac admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:21.927774 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:21.969314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c7f09-4d3f-4915-b15e-39fd24bfc7bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:21.969923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c7f09-4d3f-4915-b15e-39fd24bfc7bf service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:21.970199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c7f09-4d3f-4915-b15e-39fd24bfc7bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:21.970469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c7f09-4d3f-4915-b15e-39fd24bfc7bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:21.972202 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a00c7f09-4d3f-4915-b15e-39fd24bfc7bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:22.005095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:22.007268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:22.038396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c7f09-4d3f-4915-b15e-39fd24bfc7bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:22.039925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c7f09-4d3f-4915-b15e-39fd24bfc7bf service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:22.046612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:22.051098 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 929/1874] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:22.081285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64168362-689b-4ebf-90d6-7e087e67080a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:22.086126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:22.086774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:22.087083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:22.087469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] PATH_INFO: `/v3/projects/723bc208147e439c88e9daec8f3b1a33` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:22.088511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=723bc208147e439c88e9daec8f3b1a33)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:22.090397 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:22.141571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '723bc208147e439c88e9daec8f3b1a33', 'request_id': 'req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:22.148492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '723bc208147e439c88e9daec8f3b1a33', 'request_id': 'req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:22.156050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '723bc208147e439c88e9daec8f3b1a33', 'request_id': 'req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:22.167115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] The token cache is being invalidate because project 723bc208147e439c88e9daec8f3b1a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:22.167402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:22.168104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cb0cf02e-e46a-4c54-9857-7dcc939b3c7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:22.169842 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 946/1875] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:22 2022] DELETE /identity/v3/projects/723bc208147e439c88e9daec8f3b1a33 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:22.176328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ffe2-53c8-4101-876c-6a3f203a3e17 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:22.176620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ffe2-53c8-4101-876c-6a3f203a3e17 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:22.176897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ffe2-53c8-4101-876c-6a3f203a3e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:22.197507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-26e6ffe2-53c8-4101-876c-6a3f203a3e17 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:22.198828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-26e6ffe2-53c8-4101-876c-6a3f203a3e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:22.263914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 930/1876] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:22.274677 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978d071-569a-4193-853e-b10744c62748 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:22.274982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978d071-569a-4193-853e-b10744c62748 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:22.275298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978d071-569a-4193-853e-b10744c62748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:22.319680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8978d071-569a-4193-853e-b10744c62748 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:22.320899 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8978d071-569a-4193-853e-b10744c62748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:22.329564 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 947/1877] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:22.333271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6ffe2-53c8-4101-876c-6a3f203a3e17 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:22.343870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6ffe2-53c8-4101-876c-6a3f203a3e17 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:22.350072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20780b33-8b23-430a-ba56-3bb08f0b5bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:22.350072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20780b33-8b23-430a-ba56-3bb08f0b5bed admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:22.350072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20780b33-8b23-430a-ba56-3bb08f0b5bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:22.350072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20780b33-8b23-430a-ba56-3bb08f0b5bed admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:22.350513 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20780b33-8b23-430a-ba56-3bb08f0b5bed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:22.350741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20780b33-8b23-430a-ba56-3bb08f0b5bed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:22.352365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20780b33-8b23-430a-ba56-3bb08f0b5bed admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:22.365112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 931/1878] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:22.369705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978d071-569a-4193-853e-b10744c62748 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:22.379507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978d071-569a-4193-853e-b10744c62748 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:22.384141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda2877-ccef-4674-8f9f-9c44ff2661a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:22.384737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda2877-ccef-4674-8f9f-9c44ff2661a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:22.385014 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda2877-ccef-4674-8f9f-9c44ff2661a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:22.385284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda2877-ccef-4674-8f9f-9c44ff2661a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:22.385913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda2877-ccef-4674-8f9f-9c44ff2661a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:22.388141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda2877-ccef-4674-8f9f-9c44ff2661a3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:22.418626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 948/1879] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:22 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:22.422923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20780b33-8b23-430a-ba56-3bb08f0b5bed admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:22.432902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20780b33-8b23-430a-ba56-3bb08f0b5bed admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:22.483050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:22.489419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82980d35-52bf-4e4d-b7f8-b9f278dfb6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:22.490044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82980d35-52bf-4e4d-b7f8-b9f278dfb6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:22.490328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82980d35-52bf-4e4d-b7f8-b9f278dfb6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:22.490606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82980d35-52bf-4e4d-b7f8-b9f278dfb6e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:22.491339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82980d35-52bf-4e4d-b7f8-b9f278dfb6e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:22.493373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82980d35-52bf-4e4d-b7f8-b9f278dfb6e2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:22.517273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 932/1880] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:22 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:22.518368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda2877-ccef-4674-8f9f-9c44ff2661a3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:22.529420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda2877-ccef-4674-8f9f-9c44ff2661a3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:22.534924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d1e9a-a630-40d5-9c36-2869113cc042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:22.535668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d1e9a-a630-40d5-9c36-2869113cc042 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:22.536041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d1e9a-a630-40d5-9c36-2869113cc042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:22.536374 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d1e9a-a630-40d5-9c36-2869113cc042 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:22.537111 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d1e9a-a630-40d5-9c36-2869113cc042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:22.538752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d1e9a-a630-40d5-9c36-2869113cc042 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:22.549106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 949/1881] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:22.553145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82980d35-52bf-4e4d-b7f8-b9f278dfb6e2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:22.563393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82980d35-52bf-4e4d-b7f8-b9f278dfb6e2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:22.568891 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b75a5-7c10-4013-9651-3cc9efd337a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:22.569537 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b75a5-7c10-4013-9651-3cc9efd337a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:22.569963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b75a5-7c10-4013-9651-3cc9efd337a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:22.570249 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b75a5-7c10-4013-9651-3cc9efd337a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:22.570904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b75a5-7c10-4013-9651-3cc9efd337a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:22.572523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b75a5-7c10-4013-9651-3cc9efd337a5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:22.585791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 933/1882] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:22.590135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d1e9a-a630-40d5-9c36-2869113cc042 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:22.602011 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d1e9a-a630-40d5-9c36-2869113cc042 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:22.606853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bc3be-3d49-4c7a-be1a-fcab105f5b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:22.607571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bc3be-3d49-4c7a-be1a-fcab105f5b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:22.607849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bc3be-3d49-4c7a-be1a-fcab105f5b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:22.608123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bc3be-3d49-4c7a-be1a-fcab105f5b13 admin admin] PATH_INFO: `/v3/projects/5a691f5b28f04249bd5dde990899d96b/users/968e6fd6cf6c4f8cbc5c9c63123584fc/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:22.628993 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bc3be-3d49-4c7a-be1a-fcab105f5b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a691f5b28f04249bd5dde990899d96b, user_id=968e6fd6cf6c4f8cbc5c9c63123584fc, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:22.631914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bc3be-3d49-4c7a-be1a-fcab105f5b13 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:22.648092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 950/1883] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:22 2022] PUT /identity/v3/projects/5a691f5b28f04249bd5dde990899d96b/users/968e6fd6cf6c4f8cbc5c9c63123584fc/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:22.659177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e45e-f4bb-495b-84c4-fb811deab590 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:22.659475 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e45e-f4bb-495b-84c4-fb811deab590 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:22.659755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e45e-f4bb-495b-84c4-fb811deab590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:22.695707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0c48e45e-f4bb-495b-84c4-fb811deab590 None None] MFA Rules not processed for user `968e6fd6cf6c4f8cbc5c9c63123584fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:22.696943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0c48e45e-f4bb-495b-84c4-fb811deab590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:22.768849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 934/1884] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:22.773265 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bc3be-3d49-4c7a-be1a-fcab105f5b13 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:22.846735 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:22.850912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bc3be-3d49-4c7a-be1a-fcab105f5b13 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:22.856923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868215d8-fcbd-44b0-90f7-b1f54f73f2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:22.857551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868215d8-fcbd-44b0-90f7-b1f54f73f2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:22.857840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868215d8-fcbd-44b0-90f7-b1f54f73f2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:22.858121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868215d8-fcbd-44b0-90f7-b1f54f73f2f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:22.859031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868215d8-fcbd-44b0-90f7-b1f54f73f2f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:22.861114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868215d8-fcbd-44b0-90f7-b1f54f73f2f7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:22.891720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 951/1885] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:22 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:22.896517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48e45e-f4bb-495b-84c4-fb811deab590 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:22.908479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48e45e-f4bb-495b-84c4-fb811deab590 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:22.999635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fd859-96be-4ab1-835f-3730376c2798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:23.000272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fd859-96be-4ab1-835f-3730376c2798 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.000560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fd859-96be-4ab1-835f-3730376c2798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.000838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fd859-96be-4ab1-835f-3730376c2798 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.001491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fd859-96be-4ab1-835f-3730376c2798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:23.003487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fd859-96be-4ab1-835f-3730376c2798 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:23.028240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 935/1886] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:22 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:23.032162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868215d8-fcbd-44b0-90f7-b1f54f73f2f7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:23.043457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868215d8-fcbd-44b0-90f7-b1f54f73f2f7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:23.048377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb6f17-6b38-4cb1-ae1d-61ab5b2000fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:23.048996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb6f17-6b38-4cb1-ae1d-61ab5b2000fb admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.049267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb6f17-6b38-4cb1-ae1d-61ab5b2000fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.049879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb6f17-6b38-4cb1-ae1d-61ab5b2000fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.050506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb6f17-6b38-4cb1-ae1d-61ab5b2000fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:23.052078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb6f17-6b38-4cb1-ae1d-61ab5b2000fb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:23.064926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 952/1887] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:23.069229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fd859-96be-4ab1-835f-3730376c2798 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:23.081571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fd859-96be-4ab1-835f-3730376c2798 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:23.086881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e00791-c0ff-4d36-911f-37c50baba308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:23.087544 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e00791-c0ff-4d36-911f-37c50baba308 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.087828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e00791-c0ff-4d36-911f-37c50baba308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.088104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e00791-c0ff-4d36-911f-37c50baba308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.088966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e00791-c0ff-4d36-911f-37c50baba308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:23.090418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e00791-c0ff-4d36-911f-37c50baba308 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:23.101844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 936/1888] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:23.106012 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb6f17-6b38-4cb1-ae1d-61ab5b2000fb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:23.119417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb6f17-6b38-4cb1-ae1d-61ab5b2000fb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:23.126022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b73d97-933a-42f8-92bf-1cb7ae406036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:23.126712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b73d97-933a-42f8-92bf-1cb7ae406036 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.127238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b73d97-933a-42f8-92bf-1cb7ae406036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.127372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b73d97-933a-42f8-92bf-1cb7ae406036 admin admin] PATH_INFO: `/v3/projects/6fd044a282224973af14303ff923bd9e/users/305ba887bc2144c99497d50925f18a21/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.142787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b73d97-933a-42f8-92bf-1cb7ae406036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fd044a282224973af14303ff923bd9e, user_id=305ba887bc2144c99497d50925f18a21, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:23.145482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b73d97-933a-42f8-92bf-1cb7ae406036 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:23.158797 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 953/1889] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:23 2022] PUT /identity/v3/projects/6fd044a282224973af14303ff923bd9e/users/305ba887bc2144c99497d50925f18a21/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:23.171652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6905e1-83b5-48df-bb1b-bdf74ad8b022 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.171879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6905e1-83b5-48df-bb1b-bdf74ad8b022 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.172155 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6905e1-83b5-48df-bb1b-bdf74ad8b022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.200197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9f6905e1-83b5-48df-bb1b-bdf74ad8b022 None None] MFA Rules not processed for user `305ba887bc2144c99497d50925f18a21`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:23.201403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9f6905e1-83b5-48df-bb1b-bdf74ad8b022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:23.233197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:23.266624 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 937/1890] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:23.271103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b73d97-933a-42f8-92bf-1cb7ae406036 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:23.332241 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b73d97-933a-42f8-92bf-1cb7ae406036 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:23.336815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aea776-ba54-423e-90f6-a53871223e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:23.337421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea776-ba54-423e-90f6-a53871223e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.337703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea776-ba54-423e-90f6-a53871223e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.337978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea776-ba54-423e-90f6-a53871223e57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.338612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aea776-ba54-423e-90f6-a53871223e57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:23.340561 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aea776-ba54-423e-90f6-a53871223e57 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:23.367268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 954/1891] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:23 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:23.371462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6905e1-83b5-48df-bb1b-bdf74ad8b022 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:23.383242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6905e1-83b5-48df-bb1b-bdf74ad8b022 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:23.440856 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc04a3-f4c8-40ce-a584-875c51316fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:23.441463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc04a3-f4c8-40ce-a584-875c51316fec admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.441740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc04a3-f4c8-40ce-a584-875c51316fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.442009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc04a3-f4c8-40ce-a584-875c51316fec admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.442885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc04a3-f4c8-40ce-a584-875c51316fec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:23.444812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc04a3-f4c8-40ce-a584-875c51316fec admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:23.465794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 938/1892] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:23 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:23.470163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aea776-ba54-423e-90f6-a53871223e57 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:23.480887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aea776-ba54-423e-90f6-a53871223e57 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:23.486167 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223579a-01ff-41bf-8d95-324df993da5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:23.486771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223579a-01ff-41bf-8d95-324df993da5e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.487127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223579a-01ff-41bf-8d95-324df993da5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.487369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223579a-01ff-41bf-8d95-324df993da5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.488004 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223579a-01ff-41bf-8d95-324df993da5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:23.489596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223579a-01ff-41bf-8d95-324df993da5e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:23.501538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 955/1893] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:23.505904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc04a3-f4c8-40ce-a584-875c51316fec admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:23.516814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc04a3-f4c8-40ce-a584-875c51316fec admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:23.521657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98f92b-2851-43c5-98b0-9008eb446a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:23.522338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98f92b-2851-43c5-98b0-9008eb446a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.522623 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98f92b-2851-43c5-98b0-9008eb446a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.522900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98f92b-2851-43c5-98b0-9008eb446a21 admin admin] PATH_INFO: `/v3/projects/6e1b3104e159483a8330f399db9cfba6/users/7370381a622b48df9e6b9e885bf123d9/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.538973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98f92b-2851-43c5-98b0-9008eb446a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1b3104e159483a8330f399db9cfba6, user_id=7370381a622b48df9e6b9e885bf123d9, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:23.541816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98f92b-2851-43c5-98b0-9008eb446a21 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:23.555575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 939/1894] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:23 2022] PUT /identity/v3/projects/6e1b3104e159483a8330f399db9cfba6/users/7370381a622b48df9e6b9e885bf123d9/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:23.559661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223579a-01ff-41bf-8d95-324df993da5e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:23.569946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223579a-01ff-41bf-8d95-324df993da5e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:23.626259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d780d95-c619-4c42-aa81-5462b27ab711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:23.626800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d780d95-c619-4c42-aa81-5462b27ab711 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.627112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d780d95-c619-4c42-aa81-5462b27ab711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.627390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d780d95-c619-4c42-aa81-5462b27ab711 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.628057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d780d95-c619-4c42-aa81-5462b27ab711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:23.629649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d780d95-c619-4c42-aa81-5462b27ab711 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:23.639362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 956/1895] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:23.644331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98f92b-2851-43c5-98b0-9008eb446a21 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:23.657030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98f92b-2851-43c5-98b0-9008eb446a21 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:23.661708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dd6e4-037f-4ccc-bfa1-1af63163f695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:23.662370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dd6e4-037f-4ccc-bfa1-1af63163f695 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.662649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dd6e4-037f-4ccc-bfa1-1af63163f695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.662932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dd6e4-037f-4ccc-bfa1-1af63163f695 admin admin] PATH_INFO: `/v3/projects/6e1b3104e159483a8330f399db9cfba6/users/7370381a622b48df9e6b9e885bf123d9/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.664879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537dd6e4-037f-4ccc-bfa1-1af63163f695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1b3104e159483a8330f399db9cfba6, user_id=7370381a622b48df9e6b9e885bf123d9, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:23.667539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537dd6e4-037f-4ccc-bfa1-1af63163f695 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:23.683259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 940/1896] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:23 2022] PUT /identity/v3/projects/6e1b3104e159483a8330f399db9cfba6/users/7370381a622b48df9e6b9e885bf123d9/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:23.698256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808e718-ee13-4b43-8ed1-39db5d2ec12c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.698553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808e718-ee13-4b43-8ed1-39db5d2ec12c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.698839 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808e718-ee13-4b43-8ed1-39db5d2ec12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.728687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9808e718-ee13-4b43-8ed1-39db5d2ec12c None None] MFA Rules not processed for user `7370381a622b48df9e6b9e885bf123d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:23.729931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9808e718-ee13-4b43-8ed1-39db5d2ec12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:23.735147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:23.806480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 957/1897] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:23.819180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dd6e4-037f-4ccc-bfa1-1af63163f695 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:23.821237 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-537dd6e4-037f-4ccc-bfa1-1af63163f695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:23.899592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dd6e4-037f-4ccc-bfa1-1af63163f695 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:23.905359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e99dc6-6a69-4022-b6b3-744e8b8f7a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:23.906329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e99dc6-6a69-4022-b6b3-744e8b8f7a7b service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.906876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e99dc6-6a69-4022-b6b3-744e8b8f7a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.907402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e99dc6-6a69-4022-b6b3-744e8b8f7a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.924673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e99dc6-6a69-4022-b6b3-744e8b8f7a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:23.926658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e99dc6-6a69-4022-b6b3-744e8b8f7a7b service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:23.940851 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 941/1898] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:24.065559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808e718-ee13-4b43-8ed1-39db5d2ec12c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:24.067068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9808e718-ee13-4b43-8ed1-39db5d2ec12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:24.095126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808e718-ee13-4b43-8ed1-39db5d2ec12c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:24.107879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4bb19-c355-4662-982c-0539a7693e4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:24.109150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4bb19-c355-4662-982c-0539a7693e4e service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:24.109430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4bb19-c355-4662-982c-0539a7693e4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:24.109702 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4bb19-c355-4662-982c-0539a7693e4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:24.307943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4bb19-c355-4662-982c-0539a7693e4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:24.309853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4bb19-c355-4662-982c-0539a7693e4e service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:24.323032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 958/1899] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:25.897659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638be7c-6876-45d5-8de1-23b696e6ba10 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:25.899559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638be7c-6876-45d5-8de1-23b696e6ba10 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:25.900294 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638be7c-6876-45d5-8de1-23b696e6ba10 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:25.903226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 942/1900] 10.222.0.86 () {54 vars in 921 bytes} [Mon Aug 29 19:02:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:25.914192 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3e151-f53b-4a8d-98f0-efeb28533780 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:25.914639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3e151-f53b-4a8d-98f0-efeb28533780 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:25.914809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3e151-f53b-4a8d-98f0-efeb28533780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:25.943591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5fa3e151-f53b-4a8d-98f0-efeb28533780 None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:25.944851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5fa3e151-f53b-4a8d-98f0-efeb28533780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:26.061314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 959/1901] 10.222.0.86 () {58 vars in 1018 bytes} [Mon Aug 29 19:02:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:26.075121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638be7c-6876-45d5-8de1-23b696e6ba10 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:26.077121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e638be7c-6876-45d5-8de1-23b696e6ba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:26.097382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638be7c-6876-45d5-8de1-23b696e6ba10 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:26.103525 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b7f44-974b-4ed4-8523-366295ff85ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:26.104139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b7f44-974b-4ed4-8523-366295ff85ad service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:26.104423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b7f44-974b-4ed4-8523-366295ff85ad service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:26.104700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b7f44-974b-4ed4-8523-366295ff85ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:26.115150 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b7f44-974b-4ed4-8523-366295ff85ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:26.117066 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b7f44-974b-4ed4-8523-366295ff85ad service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:26.132081 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 943/1902] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:32.759254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3e151-f53b-4a8d-98f0-efeb28533780 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:32.794088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:32.821995 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3e151-f53b-4a8d-98f0-efeb28533780 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:32.826420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:32.827096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:32.827981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:32.828248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] PATH_INFO: `/v3/users/968e6fd6cf6c4f8cbc5c9c63123584fc` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:32.829509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=968e6fd6cf6c4f8cbc5c9c63123584fc)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:32.831490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:32.875745 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '968e6fd6cf6c4f8cbc5c9c63123584fc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:32.876387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '968e6fd6cf6c4f8cbc5c9c63123584fc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:32.888670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '968e6fd6cf6c4f8cbc5c9c63123584fc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:32.896405 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '968e6fd6cf6c4f8cbc5c9c63123584fc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:32.903546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '968e6fd6cf6c4f8cbc5c9c63123584fc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:32.909672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 960/1903] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:32 2022] DELETE /identity/v3/users/968e6fd6cf6c4f8cbc5c9c63123584fc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:32.914297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93191-28cd-47e6-8009-e43fd2412751 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:32.915178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93191-28cd-47e6-8009-e43fd2412751 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:32.915728 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93191-28cd-47e6-8009-e43fd2412751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:32.960151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a0e93191-28cd-47e6-8009-e43fd2412751 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:32.965182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a0e93191-28cd-47e6-8009-e43fd2412751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:32.994323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:33.054754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 944/1904] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:33.065196 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:33.066723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:33.144339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e219a9a-dce5-41af-9c7c-41fcdf446fc1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:33.148180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfdd38-7d8d-4964-b7af-c1b7d2cfef4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:33.148875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfdd38-7d8d-4964-b7af-c1b7d2cfef4b service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:33.149219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfdd38-7d8d-4964-b7af-c1b7d2cfef4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:33.149561 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfdd38-7d8d-4964-b7af-c1b7d2cfef4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:33.163322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfdd38-7d8d-4964-b7af-c1b7d2cfef4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:33.164843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfdd38-7d8d-4964-b7af-c1b7d2cfef4b service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:33.177486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 961/1905] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:33.635730 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93191-28cd-47e6-8009-e43fd2412751 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:33.637166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a0e93191-28cd-47e6-8009-e43fd2412751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:33.649379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93191-28cd-47e6-8009-e43fd2412751 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:33.654125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:33.654832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:33.655406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:33.655848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] PATH_INFO: `/v3/projects/5a691f5b28f04249bd5dde990899d96b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:33.657052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a691f5b28f04249bd5dde990899d96b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:33.659143 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:33.727386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a691f5b28f04249bd5dde990899d96b', 'request_id': 'req-a606e66d-0323-42da-98ef-6fbf8312d665'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:33.736271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a691f5b28f04249bd5dde990899d96b', 'request_id': 'req-a606e66d-0323-42da-98ef-6fbf8312d665'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:33.746739 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a691f5b28f04249bd5dde990899d96b', 'request_id': 'req-a606e66d-0323-42da-98ef-6fbf8312d665'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:33.760529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] The token cache is being invalidate because project 5a691f5b28f04249bd5dde990899d96b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:33.760819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:33.761536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:33.763634 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 945/1906] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:33 2022] DELETE /identity/v3/projects/5a691f5b28f04249bd5dde990899d96b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:33.768394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfdd38-7d8d-4964-b7af-c1b7d2cfef4b service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:33.781760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfdd38-7d8d-4964-b7af-c1b7d2cfef4b service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:33.845352 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:33.854513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:33.855595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:33.856135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:33.856349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] PATH_INFO: `/v3/users/305ba887bc2144c99497d50925f18a21` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:33.859181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=305ba887bc2144c99497d50925f18a21)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:33.860085 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:33.925671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305ba887bc2144c99497d50925f18a21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:33.926550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305ba887bc2144c99497d50925f18a21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:33.943024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305ba887bc2144c99497d50925f18a21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:33.951942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305ba887bc2144c99497d50925f18a21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:33.961136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305ba887bc2144c99497d50925f18a21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:33.970888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 962/1907] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:33 2022] DELETE /identity/v3/users/305ba887bc2144c99497d50925f18a21 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:33.986641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:33.988293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:34.060482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606e66d-0323-42da-98ef-6fbf8312d665 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:34.120870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f4ec3-f073-4152-ab28-8c6a2958532f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:34.121491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f4ec3-f073-4152-ab28-8c6a2958532f service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:34.121773 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f4ec3-f073-4152-ab28-8c6a2958532f service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:34.122308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f4ec3-f073-4152-ab28-8c6a2958532f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:34.124793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-176f4ec3-f073-4152-ab28-8c6a2958532f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:34.140007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:34.206905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f4ec3-f073-4152-ab28-8c6a2958532f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:34.212658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f4ec3-f073-4152-ab28-8c6a2958532f service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:34.223881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 946/1908] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:34.288604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:34.292783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:34.394512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059444e-5fbf-46b6-a59c-1fd96af44816 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:34.403124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:34.403881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:34.404349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:34.404683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] PATH_INFO: `/v3/projects/6fd044a282224973af14303ff923bd9e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:34.406801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fd044a282224973af14303ff923bd9e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:34.408569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:34.480319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fd044a282224973af14303ff923bd9e', 'request_id': 'req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:34.493156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fd044a282224973af14303ff923bd9e', 'request_id': 'req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:34.505306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fd044a282224973af14303ff923bd9e', 'request_id': 'req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:34.534742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] The token cache is being invalidate because project 6fd044a282224973af14303ff923bd9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:34.535070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:34.536003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:34.539240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 963/1909] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:34 2022] DELETE /identity/v3/projects/6fd044a282224973af14303ff923bd9e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:34.544161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f4ec3-f073-4152-ab28-8c6a2958532f service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:34.559161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f4ec3-f073-4152-ab28-8c6a2958532f service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:34.629187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f31e0-b13a-4e75-95ea-f2bdcecef7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:34.629830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f31e0-b13a-4e75-95ea-f2bdcecef7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:34.630225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f31e0-b13a-4e75-95ea-f2bdcecef7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:34.630636 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f31e0-b13a-4e75-95ea-f2bdcecef7ca admin admin] PATH_INFO: `/v3/users/7370381a622b48df9e6b9e885bf123d9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:34.632091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f31e0-b13a-4e75-95ea-f2bdcecef7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=7370381a622b48df9e6b9e885bf123d9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:34.634239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f31e0-b13a-4e75-95ea-f2bdcecef7ca admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:34.695372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-313f31e0-b13a-4e75-95ea-f2bdcecef7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7370381a622b48df9e6b9e885bf123d9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:34.712752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-313f31e0-b13a-4e75-95ea-f2bdcecef7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7370381a622b48df9e6b9e885bf123d9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:34.713929 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-313f31e0-b13a-4e75-95ea-f2bdcecef7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7370381a622b48df9e6b9e885bf123d9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:34.714867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-313f31e0-b13a-4e75-95ea-f2bdcecef7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7370381a622b48df9e6b9e885bf123d9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:34.725514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-313f31e0-b13a-4e75-95ea-f2bdcecef7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7370381a622b48df9e6b9e885bf123d9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:34.739502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 947/1910] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:34 2022] DELETE /identity/v3/users/7370381a622b48df9e6b9e885bf123d9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:35.066680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:35.071524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:35.084119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:35.091020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1e0a0-75a1-4bcc-8394-51e053a9a793 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:35.155452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:35.156158 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.156434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.156705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] PATH_INFO: `/v3/projects/6e1b3104e159483a8330f399db9cfba6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.157821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e1b3104e159483a8330f399db9cfba6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:35.159739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:35.206027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e1b3104e159483a8330f399db9cfba6', 'request_id': 'req-7a41f9cb-344d-422e-80e1-73594b6dd31b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:35.214525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e1b3104e159483a8330f399db9cfba6', 'request_id': 'req-7a41f9cb-344d-422e-80e1-73594b6dd31b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:35.221748 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e1b3104e159483a8330f399db9cfba6', 'request_id': 'req-7a41f9cb-344d-422e-80e1-73594b6dd31b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:35.235326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] The token cache is being invalidate because project 6e1b3104e159483a8330f399db9cfba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:35.235618 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:35.236471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a41f9cb-344d-422e-80e1-73594b6dd31b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:35.240481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 964/1911] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:35 2022] DELETE /identity/v3/projects/6e1b3104e159483a8330f399db9cfba6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:35.246089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279ec82-8e28-4f1f-961b-8d2c36cd44b2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.246560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279ec82-8e28-4f1f-961b-8d2c36cd44b2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.247258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279ec82-8e28-4f1f-961b-8d2c36cd44b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.274063 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6279ec82-8e28-4f1f-961b-8d2c36cd44b2 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:35.275322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6279ec82-8e28-4f1f-961b-8d2c36cd44b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:35.300608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:35.343614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 948/1912] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:35.352384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aba334-c42c-4d14-8983-5ed60e5f4dfc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.352708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aba334-c42c-4d14-8983-5ed60e5f4dfc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.352990 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aba334-c42c-4d14-8983-5ed60e5f4dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.375445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-92aba334-c42c-4d14-8983-5ed60e5f4dfc None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:35.376709 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-92aba334-c42c-4d14-8983-5ed60e5f4dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:35.390019 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 965/1913] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:35.391914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279ec82-8e28-4f1f-961b-8d2c36cd44b2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:35.410612 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279ec82-8e28-4f1f-961b-8d2c36cd44b2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:35.429600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e14db-55bd-43a0-a2b0-2bba2e4511ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:35.430248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e14db-55bd-43a0-a2b0-2bba2e4511ae admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.435206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e14db-55bd-43a0-a2b0-2bba2e4511ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.435823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e14db-55bd-43a0-a2b0-2bba2e4511ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.437761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e14db-55bd-43a0-a2b0-2bba2e4511ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:35.438240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e14db-55bd-43a0-a2b0-2bba2e4511ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:35.440177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e14db-55bd-43a0-a2b0-2bba2e4511ae admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:35.469713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 949/1914] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:35.469998 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aba334-c42c-4d14-8983-5ed60e5f4dfc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:35.489743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aba334-c42c-4d14-8983-5ed60e5f4dfc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:35.526814 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853991e6-d428-4d41-99ff-b2df23a5a7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:35.528053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853991e6-d428-4d41-99ff-b2df23a5a7be admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.528473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853991e6-d428-4d41-99ff-b2df23a5a7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.528891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853991e6-d428-4d41-99ff-b2df23a5a7be admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.529556 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853991e6-d428-4d41-99ff-b2df23a5a7be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:35.536000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853991e6-d428-4d41-99ff-b2df23a5a7be admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:35.566551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e14db-55bd-43a0-a2b0-2bba2e4511ae admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:35.568931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-645e14db-55bd-43a0-a2b0-2bba2e4511ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:35.591321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e14db-55bd-43a0-a2b0-2bba2e4511ae admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:35.609040 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 966/1915] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:35 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:35.610152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853991e6-d428-4d41-99ff-b2df23a5a7be admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:35.611132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16270feb-4b7a-4a9d-a13d-c2f9e0806323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:35.612138 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16270feb-4b7a-4a9d-a13d-c2f9e0806323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.612782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16270feb-4b7a-4a9d-a13d-c2f9e0806323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.613266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16270feb-4b7a-4a9d-a13d-c2f9e0806323 admin admin] PATH_INFO: `/v3/users/4e4179e482d94a3eb707491c442cfd7c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.614930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16270feb-4b7a-4a9d-a13d-c2f9e0806323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e4179e482d94a3eb707491c442cfd7c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:35.621011 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16270feb-4b7a-4a9d-a13d-c2f9e0806323 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:35.712450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-16270feb-4b7a-4a9d-a13d-c2f9e0806323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e4179e482d94a3eb707491c442cfd7c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:35.731218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-16270feb-4b7a-4a9d-a13d-c2f9e0806323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e4179e482d94a3eb707491c442cfd7c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:35.735502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853991e6-d428-4d41-99ff-b2df23a5a7be admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:35.735677 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-16270feb-4b7a-4a9d-a13d-c2f9e0806323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e4179e482d94a3eb707491c442cfd7c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:35.736613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-16270feb-4b7a-4a9d-a13d-c2f9e0806323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e4179e482d94a3eb707491c442cfd7c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:35.742643 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe969ef-8c47-4469-853f-90a597c3e327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:35.743724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe969ef-8c47-4469-853f-90a597c3e327 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.744084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe969ef-8c47-4469-853f-90a597c3e327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.744361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe969ef-8c47-4469-853f-90a597c3e327 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.745146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe969ef-8c47-4469-853f-90a597c3e327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:35.747435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe969ef-8c47-4469-853f-90a597c3e327 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:35.750181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-16270feb-4b7a-4a9d-a13d-c2f9e0806323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e4179e482d94a3eb707491c442cfd7c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:35.766904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:35.768172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 950/1916] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:35 2022] DELETE /identity/v3/users/4e4179e482d94a3eb707491c442cfd7c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:35.773285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2a230-ff63-4303-a3e5-3f67260139e0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.773620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2a230-ff63-4303-a3e5-3f67260139e0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.773897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2a230-ff63-4303-a3e5-3f67260139e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.793028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 967/1917] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:35 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:35.796823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe969ef-8c47-4469-853f-90a597c3e327 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:35.799504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ebe969ef-8c47-4469-853f-90a597c3e327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:35.802770 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-63a2a230-ff63-4303-a3e5-3f67260139e0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:35.804186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-63a2a230-ff63-4303-a3e5-3f67260139e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:35.813376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe969ef-8c47-4469-853f-90a597c3e327 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:35.895247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791d9c9-a5bb-481e-bbc9-842a020e2bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:35.896013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791d9c9-a5bb-481e-bbc9-842a020e2bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.896301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791d9c9-a5bb-481e-bbc9-842a020e2bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.896617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791d9c9-a5bb-481e-bbc9-842a020e2bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.897650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791d9c9-a5bb-481e-bbc9-842a020e2bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:35.899644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791d9c9-a5bb-481e-bbc9-842a020e2bb9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:35.909411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 951/1918] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:35.919828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2a230-ff63-4303-a3e5-3f67260139e0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:35.921723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 968/1919] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:35.922524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-63a2a230-ff63-4303-a3e5-3f67260139e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:35.926133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791d9c9-a5bb-481e-bbc9-842a020e2bb9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:35.940856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791d9c9-a5bb-481e-bbc9-842a020e2bb9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:35.947317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c74f6d-5b16-4fe4-ba82-d6fa8efb5f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:35.947983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c74f6d-5b16-4fe4-ba82-d6fa8efb5f6f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.948781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c74f6d-5b16-4fe4-ba82-d6fa8efb5f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.949268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c74f6d-5b16-4fe4-ba82-d6fa8efb5f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.950202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c74f6d-5b16-4fe4-ba82-d6fa8efb5f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:35.952006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c74f6d-5b16-4fe4-ba82-d6fa8efb5f6f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:35.965392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 969/1920] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:35.970405 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c74f6d-5b16-4fe4-ba82-d6fa8efb5f6f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:35.975924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:35.986504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c74f6d-5b16-4fe4-ba82-d6fa8efb5f6f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:35.995692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22f99c-b9c2-47c3-b3ff-3950becf8388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:35.996767 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22f99c-b9c2-47c3-b3ff-3950becf8388 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.997317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22f99c-b9c2-47c3-b3ff-3950becf8388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.997792 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22f99c-b9c2-47c3-b3ff-3950becf8388 admin admin] PATH_INFO: `/v3/projects/228b1c08250e4d138423d1e0a9324ff5/users/f18ff98ad4e24ddd9089917c7d081acd/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:36.021981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2a230-ff63-4303-a3e5-3f67260139e0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:36.038125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22f99c-b9c2-47c3-b3ff-3950becf8388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=228b1c08250e4d138423d1e0a9324ff5, user_id=f18ff98ad4e24ddd9089917c7d081acd, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:36.043314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22f99c-b9c2-47c3-b3ff-3950becf8388 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:36.064541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 970/1921] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:35 2022] PUT /identity/v3/projects/228b1c08250e4d138423d1e0a9324ff5/users/f18ff98ad4e24ddd9089917c7d081acd/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:36.068809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7df8d5-7c9f-40e3-b6d8-db2b6a25bc38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:36.069457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7df8d5-7c9f-40e3-b6d8-db2b6a25bc38 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:36.069738 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7df8d5-7c9f-40e3-b6d8-db2b6a25bc38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:36.070009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7df8d5-7c9f-40e3-b6d8-db2b6a25bc38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:36.085752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be0314-9f65-402b-871d-9caec983b052 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:36.086245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be0314-9f65-402b-871d-9caec983b052 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:36.086528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be0314-9f65-402b-871d-9caec983b052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:36.104886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7df8d5-7c9f-40e3-b6d8-db2b6a25bc38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:36.109795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7df8d5-7c9f-40e3-b6d8-db2b6a25bc38 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:36.156657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 952/1922] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:02:36.217263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-47be0314-9f65-402b-871d-9caec983b052 None None] MFA Rules not processed for user `f18ff98ad4e24ddd9089917c7d081acd`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:36.228525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-47be0314-9f65-402b-871d-9caec983b052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:36.392780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 971/1923] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:36.396961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7df8d5-7c9f-40e3-b6d8-db2b6a25bc38 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:36.412705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7df8d5-7c9f-40e3-b6d8-db2b6a25bc38 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:36.588124 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00969d77-15f2-42cc-b651-ae27f868f5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:36.588615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00969d77-15f2-42cc-b651-ae27f868f5de admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:36.588896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00969d77-15f2-42cc-b651-ae27f868f5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:36.589166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00969d77-15f2-42cc-b651-ae27f868f5de admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:36.589797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00969d77-15f2-42cc-b651-ae27f868f5de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:36.591937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00969d77-15f2-42cc-b651-ae27f868f5de admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:36.636212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 953/1924] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:36 2022] POST /identity/v3/projects => generated 383 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:36.641062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be0314-9f65-402b-871d-9caec983b052 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:36.659304 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be0314-9f65-402b-871d-9caec983b052 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:36.752183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:36.773675 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af18a7-df80-49d7-90ec-36cf002de729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:36.774976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af18a7-df80-49d7-90ec-36cf002de729 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:36.775687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af18a7-df80-49d7-90ec-36cf002de729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:36.776003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af18a7-df80-49d7-90ec-36cf002de729 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:36.776780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af18a7-df80-49d7-90ec-36cf002de729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:36.781296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af18a7-df80-49d7-90ec-36cf002de729 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:36.815576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 972/1925] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:36 2022] POST /identity/v3/users => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:36.819516 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00969d77-15f2-42cc-b651-ae27f868f5de admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:36.836222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00969d77-15f2-42cc-b651-ae27f868f5de admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:36.853445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e475e-68c1-4776-a50d-afcfa1e7b765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:36.854323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e475e-68c1-4776-a50d-afcfa1e7b765 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:36.854490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e475e-68c1-4776-a50d-afcfa1e7b765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:36.854976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e475e-68c1-4776-a50d-afcfa1e7b765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:36.855455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e475e-68c1-4776-a50d-afcfa1e7b765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:36.857070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e475e-68c1-4776-a50d-afcfa1e7b765 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:36.883109 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 954/1926] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:36.888238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af18a7-df80-49d7-90ec-36cf002de729 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:36.915468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af18a7-df80-49d7-90ec-36cf002de729 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:36.934637 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14527e2-72d8-4fd7-8fb4-c29201709c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:36.935355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14527e2-72d8-4fd7-8fb4-c29201709c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:36.935634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14527e2-72d8-4fd7-8fb4-c29201709c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:36.936688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14527e2-72d8-4fd7-8fb4-c29201709c95 admin admin] PATH_INFO: `/v3/projects/98c18342670c4903b9b3bb27fec48a28/users/173f14e03de24dc6a65f3bc89ce926db/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:36.981779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14527e2-72d8-4fd7-8fb4-c29201709c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c18342670c4903b9b3bb27fec48a28, user_id=173f14e03de24dc6a65f3bc89ce926db, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:36.987669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14527e2-72d8-4fd7-8fb4-c29201709c95 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:37.011870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e475e-68c1-4776-a50d-afcfa1e7b765 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:37.014276 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-653e475e-68c1-4776-a50d-afcfa1e7b765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.028569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 973/1927] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:36 2022] PUT /identity/v3/projects/98c18342670c4903b9b3bb27fec48a28/users/173f14e03de24dc6a65f3bc89ce926db/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:37.029724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14527e2-72d8-4fd7-8fb4-c29201709c95 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:37.042386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e475e-68c1-4776-a50d-afcfa1e7b765 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:37.197344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14527e2-72d8-4fd7-8fb4-c29201709c95 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:37.225905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e5c6d-2dc9-4937-b39d-11b816a45f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:37.226747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e5c6d-2dc9-4937-b39d-11b816a45f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.227240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e5c6d-2dc9-4937-b39d-11b816a45f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.228661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e5c6d-2dc9-4937-b39d-11b816a45f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.229600 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e5c6d-2dc9-4937-b39d-11b816a45f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:37.232747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e5c6d-2dc9-4937-b39d-11b816a45f6b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:37.242525 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:37.246013 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.246983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.247198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] PATH_INFO: `/v3/users/b97684f6fd344708a0634f467a561d02` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.251278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b97684f6fd344708a0634f467a561d02)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:37.255972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:37.259628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 974/1928] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:37.261197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e5c6d-2dc9-4937-b39d-11b816a45f6b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:37.273205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e5c6d-2dc9-4937-b39d-11b816a45f6b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:37.282038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1b6e3-158d-4396-849a-12218364b86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:37.282949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1b6e3-158d-4396-849a-12218364b86a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.283343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1b6e3-158d-4396-849a-12218364b86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.283687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1b6e3-158d-4396-849a-12218364b86a admin admin] PATH_INFO: `/v3/projects/98c18342670c4903b9b3bb27fec48a28/users/173f14e03de24dc6a65f3bc89ce926db/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.285820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1b6e3-158d-4396-849a-12218364b86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c18342670c4903b9b3bb27fec48a28, user_id=173f14e03de24dc6a65f3bc89ce926db, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:37.288600 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1b6e3-158d-4396-849a-12218364b86a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:37.305865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 975/1929] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:37 2022] PUT /identity/v3/projects/98c18342670c4903b9b3bb27fec48a28/users/173f14e03de24dc6a65f3bc89ce926db/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:37.316418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f732-51cf-4d17-8b03-6d0c9fcf30a5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.316698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f732-51cf-4d17-8b03-6d0c9fcf30a5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.318053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f732-51cf-4d17-8b03-6d0c9fcf30a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.335962 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b97684f6fd344708a0634f467a561d02'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:37.338590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:37.355418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b97684f6fd344708a0634f467a561d02'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:37.356507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b97684f6fd344708a0634f467a561d02'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:37.357265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b97684f6fd344708a0634f467a561d02'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:37.357689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0d63f732-51cf-4d17-8b03-6d0c9fcf30a5 None None] MFA Rules not processed for user `173f14e03de24dc6a65f3bc89ce926db`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:37.359567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0d63f732-51cf-4d17-8b03-6d0c9fcf30a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.367946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b97684f6fd344708a0634f467a561d02'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:37.386050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 955/1930] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:37 2022] DELETE /identity/v3/users/b97684f6fd344708a0634f467a561d02 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:37.386050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:37.387677 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.462353 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 976/1931] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:37.464279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e951a-4ea3-4b8c-a732-a26cf0500103 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.464279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e951a-4ea3-4b8c-a732-a26cf0500103 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.464547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e951a-4ea3-4b8c-a732-a26cf0500103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.473929 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b184fc-71c8-46c9-8eb1-9743d384a29c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:37.479619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:37.480411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.480694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.481102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] PATH_INFO: `/v3/projects/3c8ee98b32ef4d18b37fda8dc8bf9b63` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.482282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c8ee98b32ef4d18b37fda8dc8bf9b63)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:37.484838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:37.494205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8e1e951a-4ea3-4b8c-a732-a26cf0500103 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:37.495922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8e1e951a-4ea3-4b8c-a732-a26cf0500103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.504963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 977/1932] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:37.505960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e951a-4ea3-4b8c-a732-a26cf0500103 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:37.507252 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8e1e951a-4ea3-4b8c-a732-a26cf0500103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.522341 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:37.539222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c8ee98b32ef4d18b37fda8dc8bf9b63', 'request_id': 'req-5171376a-623d-4121-8428-113164b4ec54'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:37.545375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c8ee98b32ef4d18b37fda8dc8bf9b63', 'request_id': 'req-5171376a-623d-4121-8428-113164b4ec54'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:37.551312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c8ee98b32ef4d18b37fda8dc8bf9b63', 'request_id': 'req-5171376a-623d-4121-8428-113164b4ec54'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:37.564569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] The token cache is being invalidate because project 3c8ee98b32ef4d18b37fda8dc8bf9b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:37.564864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:37.565513 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:37.567042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 956/1933] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:37 2022] DELETE /identity/v3/projects/3c8ee98b32ef4d18b37fda8dc8bf9b63 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:37.568058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:37.570938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e951a-4ea3-4b8c-a732-a26cf0500103 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:37.571653 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.620176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4ec23-76ca-4a2c-adbe-5218fdd7bf9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:37.620795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4ec23-76ca-4a2c-adbe-5218fdd7bf9a service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.621076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4ec23-76ca-4a2c-adbe-5218fdd7bf9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.621349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4ec23-76ca-4a2c-adbe-5218fdd7bf9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.623141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-05c4ec23-76ca-4a2c-adbe-5218fdd7bf9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.629229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171376a-623d-4121-8428-113164b4ec54 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:37.633105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d416211-bea9-4dea-bde1-977c947b77f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:37.633781 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d416211-bea9-4dea-bde1-977c947b77f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.634121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d416211-bea9-4dea-bde1-977c947b77f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.634459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d416211-bea9-4dea-bde1-977c947b77f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.636438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6d416211-bea9-4dea-bde1-977c947b77f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.653164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:37.718245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4ec23-76ca-4a2c-adbe-5218fdd7bf9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:37.720825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4ec23-76ca-4a2c-adbe-5218fdd7bf9a service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:37.733892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 978/1934] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:37.735377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4017dd-2f12-4a34-becb-394464c6098f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.735663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4017dd-2f12-4a34-becb-394464c6098f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.735936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4017dd-2f12-4a34-becb-394464c6098f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.743826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d416211-bea9-4dea-bde1-977c947b77f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:37.746658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d416211-bea9-4dea-bde1-977c947b77f6 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:37.757489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cb4017dd-2f12-4a34-becb-394464c6098f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:37.758708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cb4017dd-2f12-4a34-becb-394464c6098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.759270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 957/1935] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:37.768083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 979/1936] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:37.777325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450eeb40-74d8-4e8b-b1ae-526948194740 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.777689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450eeb40-74d8-4e8b-b1ae-526948194740 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.778053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450eeb40-74d8-4e8b-b1ae-526948194740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.803792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-450eeb40-74d8-4e8b-b1ae-526948194740 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:37.805091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-450eeb40-74d8-4e8b-b1ae-526948194740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.816096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 958/1937] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:37.820526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4017dd-2f12-4a34-becb-394464c6098f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:37.833350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4017dd-2f12-4a34-becb-394464c6098f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:37.839169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f16149-865a-4dee-853b-a5a8e98cb2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:37.839357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450eeb40-74d8-4e8b-b1ae-526948194740 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:37.839635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f16149-865a-4dee-853b-a5a8e98cb2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.840190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f16149-865a-4dee-853b-a5a8e98cb2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.840584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f16149-865a-4dee-853b-a5a8e98cb2e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.841474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f16149-865a-4dee-853b-a5a8e98cb2e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:37.841785 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f16149-865a-4dee-853b-a5a8e98cb2e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:37.844124 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-450eeb40-74d8-4e8b-b1ae-526948194740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.844247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f16149-865a-4dee-853b-a5a8e98cb2e5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:37.863039 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 980/1938] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:37.864624 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450eeb40-74d8-4e8b-b1ae-526948194740 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:37.866648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f16149-865a-4dee-853b-a5a8e98cb2e5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:37.868994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32adcd-0ebb-46a8-bd9d-f8b593a7acd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:37.869692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32adcd-0ebb-46a8-bd9d-f8b593a7acd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.869979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32adcd-0ebb-46a8-bd9d-f8b593a7acd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.870253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32adcd-0ebb-46a8-bd9d-f8b593a7acd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.872158 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-db32adcd-0ebb-46a8-bd9d-f8b593a7acd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.886652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f16149-865a-4dee-853b-a5a8e98cb2e5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:37.894554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733ab2f-e10a-4d5e-a362-f0daceaa4442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:37.895950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733ab2f-e10a-4d5e-a362-f0daceaa4442 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.896640 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733ab2f-e10a-4d5e-a362-f0daceaa4442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.897082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733ab2f-e10a-4d5e-a362-f0daceaa4442 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.898440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733ab2f-e10a-4d5e-a362-f0daceaa4442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:37.900450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733ab2f-e10a-4d5e-a362-f0daceaa4442 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:37.941140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 981/1939] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:37 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:37.959766 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733ab2f-e10a-4d5e-a362-f0daceaa4442 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:37.963562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32adcd-0ebb-46a8-bd9d-f8b593a7acd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:37.965422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32adcd-0ebb-46a8-bd9d-f8b593a7acd3 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:37.978370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 959/1940] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:38.054358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:38.084708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733ab2f-e10a-4d5e-a362-f0daceaa4442 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.090001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48261c70-df2d-4352-a283-64e7b04fbe45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.090751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48261c70-df2d-4352-a283-64e7b04fbe45 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.091322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48261c70-df2d-4352-a283-64e7b04fbe45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.091668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48261c70-df2d-4352-a283-64e7b04fbe45 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.092371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48261c70-df2d-4352-a283-64e7b04fbe45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:38.101935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48261c70-df2d-4352-a283-64e7b04fbe45 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:38.132708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32adcd-0ebb-46a8-bd9d-f8b593a7acd3 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.139779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 982/1941] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:37 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:38.146499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32adcd-0ebb-46a8-bd9d-f8b593a7acd3 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.153194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f66ca5-fe9f-4008-8448-bf66a24cf416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.153846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f66ca5-fe9f-4008-8448-bf66a24cf416 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.154128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f66ca5-fe9f-4008-8448-bf66a24cf416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.154425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f66ca5-fe9f-4008-8448-bf66a24cf416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.155173 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f66ca5-fe9f-4008-8448-bf66a24cf416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:38.156822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f66ca5-fe9f-4008-8448-bf66a24cf416 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:38.169545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 960/1942] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:38.174187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48261c70-df2d-4352-a283-64e7b04fbe45 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.188788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48261c70-df2d-4352-a283-64e7b04fbe45 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.195685 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53061b28-f689-40f4-877e-a7ea08f4f50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.196500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53061b28-f689-40f4-877e-a7ea08f4f50f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.196926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53061b28-f689-40f4-877e-a7ea08f4f50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.197368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53061b28-f689-40f4-877e-a7ea08f4f50f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.198890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53061b28-f689-40f4-877e-a7ea08f4f50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:38.200670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53061b28-f689-40f4-877e-a7ea08f4f50f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:38.219872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f66ca5-fe9f-4008-8448-bf66a24cf416 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.220571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 983/1943] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:38.231602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f66ca5-fe9f-4008-8448-bf66a24cf416 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.237908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331c6d7-c2be-4365-ae38-09e8cd73d7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.238595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c6d7-c2be-4365-ae38-09e8cd73d7b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.238894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c6d7-c2be-4365-ae38-09e8cd73d7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.239199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c6d7-c2be-4365-ae38-09e8cd73d7b5 admin admin] PATH_INFO: `/v3/projects/b9e7b8f5afff4a518c2252d29001ebaf/users/217b8c643ed94ec38304d3fcc8519fbc/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.261437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331c6d7-c2be-4365-ae38-09e8cd73d7b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e7b8f5afff4a518c2252d29001ebaf, user_id=217b8c643ed94ec38304d3fcc8519fbc, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:38.270847 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331c6d7-c2be-4365-ae38-09e8cd73d7b5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:38.296484 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 961/1944] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:38 2022] PUT /identity/v3/projects/b9e7b8f5afff4a518c2252d29001ebaf/users/217b8c643ed94ec38304d3fcc8519fbc/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:38.305531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53061b28-f689-40f4-877e-a7ea08f4f50f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.307544 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-53061b28-f689-40f4-877e-a7ea08f4f50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:38.318005 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cfd2f-e67c-480f-b8ce-d20e6bc857fb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.319666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cfd2f-e67c-480f-b8ce-d20e6bc857fb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.320523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cfd2f-e67c-480f-b8ce-d20e6bc857fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.334830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53061b28-f689-40f4-877e-a7ea08f4f50f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.385568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-428cfd2f-e67c-480f-b8ce-d20e6bc857fb None None] MFA Rules not processed for user `217b8c643ed94ec38304d3fcc8519fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:38.387051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-428cfd2f-e67c-480f-b8ce-d20e6bc857fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:38.415539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.417528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.417528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.417528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] PATH_INFO: `/v3/projects/9693a11e6b5a42acbe830515f584484e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.418586 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9693a11e6b5a42acbe830515f584484e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:38.422258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:38.447527 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:38.471419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 962/1945] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:38.475631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cfd2f-e67c-480f-b8ce-d20e6bc857fb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.477002 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-428cfd2f-e67c-480f-b8ce-d20e6bc857fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:38.487853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9693a11e6b5a42acbe830515f584484e', 'request_id': 'req-dd22d833-277c-4d1a-81d2-518bbf62726d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:38.491503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cfd2f-e67c-480f-b8ce-d20e6bc857fb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.513566 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9693a11e6b5a42acbe830515f584484e', 'request_id': 'req-dd22d833-277c-4d1a-81d2-518bbf62726d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:38.528379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9693a11e6b5a42acbe830515f584484e', 'request_id': 'req-dd22d833-277c-4d1a-81d2-518bbf62726d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:38.557576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] The token cache is being invalidate because project 9693a11e6b5a42acbe830515f584484e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:38.557867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:38.558661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:38.560499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 984/1946] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:38 2022] DELETE /identity/v3/projects/9693a11e6b5a42acbe830515f584484e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:38.561033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.562461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:38.565044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.565740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.566019 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.566290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] PATH_INFO: `/v3/users/f18ff98ad4e24ddd9089917c7d081acd` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.569623 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f18ff98ad4e24ddd9089917c7d081acd)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:38.573733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:38.644099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:38.652172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22d833-277c-4d1a-81d2-518bbf62726d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.654455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f18ff98ad4e24ddd9089917c7d081acd'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:38.660644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a5066-b8b9-4bb1-9e28-4bdd0e33ec4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.661248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a5066-b8b9-4bb1-9e28-4bdd0e33ec4b service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.661951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a5066-b8b9-4bb1-9e28-4bdd0e33ec4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.662299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a5066-b8b9-4bb1-9e28-4bdd0e33ec4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.667302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f64a5066-b8b9-4bb1-9e28-4bdd0e33ec4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:38.694524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f18ff98ad4e24ddd9089917c7d081acd'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:38.698543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f18ff98ad4e24ddd9089917c7d081acd'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:38.699251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f18ff98ad4e24ddd9089917c7d081acd'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:38.724618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f18ff98ad4e24ddd9089917c7d081acd'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:38.748296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 963/1947] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:38 2022] DELETE /identity/v3/users/f18ff98ad4e24ddd9089917c7d081acd => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:38.753760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.758813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:38.804593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a5066-b8b9-4bb1-9e28-4bdd0e33ec4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:38.808871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a5066-b8b9-4bb1-9e28-4bdd0e33ec4b service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:38.821666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 985/1948] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:38.834022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b15abd-17dd-413a-adf6-babf7782eaa5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.836507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b15abd-17dd-413a-adf6-babf7782eaa5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.837484 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b15abd-17dd-413a-adf6-babf7782eaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.865997 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-72b15abd-17dd-413a-adf6-babf7782eaa5 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:38.868915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-72b15abd-17dd-413a-adf6-babf7782eaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:38.878643 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f977ed-3a00-4c63-85f4-457d25c9492f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.885302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 986/1949] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:38.888016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.888683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.888964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.889239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] PATH_INFO: `/v3/users/b5a8883367d54071bedb46733891c03a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.891482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5a8883367d54071bedb46733891c03a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:38.892638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b15abd-17dd-413a-adf6-babf7782eaa5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.896710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:38.897701 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-72b15abd-17dd-413a-adf6-babf7782eaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:38.922184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:38.991902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b15abd-17dd-413a-adf6-babf7782eaa5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.997596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9391ca-0e9b-498b-9140-29c938b08cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.998313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9391ca-0e9b-498b-9140-29c938b08cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.998580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9391ca-0e9b-498b-9140-29c938b08cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.998987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9391ca-0e9b-498b-9140-29c938b08cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:39.015808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9391ca-0e9b-498b-9140-29c938b08cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:39.017913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9391ca-0e9b-498b-9140-29c938b08cfb service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:39.018092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5a8883367d54071bedb46733891c03a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.032808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 987/1950] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:39.041273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5a8883367d54071bedb46733891c03a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.042042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5a8883367d54071bedb46733891c03a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.042691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5a8883367d54071bedb46733891c03a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.062633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5a8883367d54071bedb46733891c03a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.082810 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 964/1951] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:38 2022] DELETE /identity/v3/users/b5a8883367d54071bedb46733891c03a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:39.437826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9391ca-0e9b-498b-9140-29c938b08cfb service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:39.439378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ff9391ca-0e9b-498b-9140-29c938b08cfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:39.455407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9391ca-0e9b-498b-9140-29c938b08cfb service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:39.513871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:39.514511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:39.514888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:39.515218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] PATH_INFO: `/v3/projects/228b1c08250e4d138423d1e0a9324ff5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:39.516353 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=228b1c08250e4d138423d1e0a9324ff5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:39.518303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:39.536140 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:39.537650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:39.553711 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efed6df-c40f-4053-92f6-f7a0a23f70a1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:39.558516 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:39.559183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:39.559466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:39.559737 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] PATH_INFO: `/v3/projects/73ca9ab5e4624ea3aabba4d5df2300e2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:39.560930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] RBAC: Authorizing `identity:delete_project(project_id=73ca9ab5e4624ea3aabba4d5df2300e2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:39.563538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:39.580563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:39.588839 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '228b1c08250e4d138423d1e0a9324ff5', 'request_id': 'req-1e367526-6c49-4468-844e-830bebe46e15'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.596887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '228b1c08250e4d138423d1e0a9324ff5', 'request_id': 'req-1e367526-6c49-4468-844e-830bebe46e15'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.603533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '228b1c08250e4d138423d1e0a9324ff5', 'request_id': 'req-1e367526-6c49-4468-844e-830bebe46e15'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.613974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ca9ab5e4624ea3aabba4d5df2300e2', 'request_id': 'req-ac16296b-bded-49bf-802c-6b28788b179c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.614876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] The token cache is being invalidate because project 228b1c08250e4d138423d1e0a9324ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:39.615194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.616077 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.617571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 988/1952] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:39 2022] DELETE /identity/v3/projects/228b1c08250e4d138423d1e0a9324ff5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:39.620787 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ca9ab5e4624ea3aabba4d5df2300e2', 'request_id': 'req-ac16296b-bded-49bf-802c-6b28788b179c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.621434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:39.622880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:39.627230 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ca9ab5e4624ea3aabba4d5df2300e2', 'request_id': 'req-ac16296b-bded-49bf-802c-6b28788b179c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.640131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] The token cache is being invalidate because project 73ca9ab5e4624ea3aabba4d5df2300e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:39.640422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.641049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.642797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 965/1953] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:39 2022] DELETE /identity/v3/projects/73ca9ab5e4624ea3aabba4d5df2300e2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:39.646501 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:39.647864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:39.689827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e367526-6c49-4468-844e-830bebe46e15 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:39.726310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16296b-bded-49bf-802c-6b28788b179c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:39.731032 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:39.731671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:39.731954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:39.732233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] PATH_INFO: `/v3/users/5ac174cfb0c549bab4c327d0875e99a3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:39.733572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ac174cfb0c549bab4c327d0875e99a3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:39.735550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:39.750239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:39.751504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:39.752235 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:39.752562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:39.752844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] PATH_INFO: `/v3/users/173f14e03de24dc6a65f3bc89ce926db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:39.753994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=173f14e03de24dc6a65f3bc89ce926db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:39.756149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:39.787339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ac174cfb0c549bab4c327d0875e99a3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.800177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ac174cfb0c549bab4c327d0875e99a3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.800877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ac174cfb0c549bab4c327d0875e99a3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.801554 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ac174cfb0c549bab4c327d0875e99a3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.807241 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '173f14e03de24dc6a65f3bc89ce926db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.808072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '173f14e03de24dc6a65f3bc89ce926db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.809909 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ac174cfb0c549bab4c327d0875e99a3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.823070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '173f14e03de24dc6a65f3bc89ce926db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.825444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 966/1954] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:39 2022] DELETE /identity/v3/users/5ac174cfb0c549bab4c327d0875e99a3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:39.834284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '173f14e03de24dc6a65f3bc89ce926db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.840041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:39.847108 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '173f14e03de24dc6a65f3bc89ce926db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:39.853936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 989/1955] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:39 2022] DELETE /identity/v3/users/173f14e03de24dc6a65f3bc89ce926db => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:40.263646 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:40.265656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:40.430419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a02252-e0c2-4575-943d-77e4d8a7f2ef admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:40.558250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:40.561358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:40.565398 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:40.566063 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.566341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.566797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] PATH_INFO: `/v3/projects/4c65cc0ed5e14982942f9ecb3aab8cf9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:40.568412 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c65cc0ed5e14982942f9ecb3aab8cf9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:40.570336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:40.592083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02694c30-f87b-44f3-835e-d5c12d86d977 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:40.599292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:40.600178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.600620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.600976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] PATH_INFO: `/v3/projects/98c18342670c4903b9b3bb27fec48a28` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:40.602481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=98c18342670c4903b9b3bb27fec48a28)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:40.604594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:40.621088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:40.629934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c65cc0ed5e14982942f9ecb3aab8cf9', 'request_id': 'req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:40.637929 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c65cc0ed5e14982942f9ecb3aab8cf9', 'request_id': 'req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:40.646591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c65cc0ed5e14982942f9ecb3aab8cf9', 'request_id': 'req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:40.663615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] The token cache is being invalidate because project 4c65cc0ed5e14982942f9ecb3aab8cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:40.664092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:40.665014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5729c886-d548-4c06-a6a5-e2dfbbdbde6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:40.665472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98c18342670c4903b9b3bb27fec48a28', 'request_id': 'req-e0e6c963-6223-4909-9481-34cbc0ddd6ed'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:40.667065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 967/1956] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:40 2022] DELETE /identity/v3/projects/4c65cc0ed5e14982942f9ecb3aab8cf9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:40.673347 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ce80-8365-46c0-91c9-0761c1ac64eb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.673806 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ce80-8365-46c0-91c9-0761c1ac64eb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.673976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ce80-8365-46c0-91c9-0761c1ac64eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:40.674217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98c18342670c4903b9b3bb27fec48a28', 'request_id': 'req-e0e6c963-6223-4909-9481-34cbc0ddd6ed'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:40.681806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98c18342670c4903b9b3bb27fec48a28', 'request_id': 'req-e0e6c963-6223-4909-9481-34cbc0ddd6ed'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:40.697071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] The token cache is being invalidate because project 98c18342670c4903b9b3bb27fec48a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:40.697447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:40.698018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0e6c963-6223-4909-9481-34cbc0ddd6ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:40.699270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a334ce80-8365-46c0-91c9-0761c1ac64eb None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:40.699815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 990/1957] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:40 2022] DELETE /identity/v3/projects/98c18342670c4903b9b3bb27fec48a28 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:40.704358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a334ce80-8365-46c0-91c9-0761c1ac64eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:40.704996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3aaf3-1bbe-45f0-8e91-ef0ed24576ad None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.705463 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3aaf3-1bbe-45f0-8e91-ef0ed24576ad None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.706318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3aaf3-1bbe-45f0-8e91-ef0ed24576ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:40.736406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-39a3aaf3-1bbe-45f0-8e91-ef0ed24576ad None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:40.737577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-39a3aaf3-1bbe-45f0-8e91-ef0ed24576ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:40.805796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 991/1958] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:40.807212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 968/1959] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:40.815358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09ecd8-1f51-41da-9a9b-211545b6a45f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.815761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09ecd8-1f51-41da-9a9b-211545b6a45f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.816119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09ecd8-1f51-41da-9a9b-211545b6a45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:40.817265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23117267-57fc-492c-b61d-a541844389d0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.817565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23117267-57fc-492c-b61d-a541844389d0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.818177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23117267-57fc-492c-b61d-a541844389d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:40.840553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9d09ecd8-1f51-41da-9a9b-211545b6a45f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:40.841772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9d09ecd8-1f51-41da-9a9b-211545b6a45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:40.844498 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-23117267-57fc-492c-b61d-a541844389d0 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:40.846010 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-23117267-57fc-492c-b61d-a541844389d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:40.856109 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 992/1960] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:40.858056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09ecd8-1f51-41da-9a9b-211545b6a45f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:40.860722 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 969/1961] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:40.865786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23117267-57fc-492c-b61d-a541844389d0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:40.870717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09ecd8-1f51-41da-9a9b-211545b6a45f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:40.876407 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23117267-57fc-492c-b61d-a541844389d0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:40.876973 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8536327-4bf6-484e-b0ee-71cbf42044e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:40.878466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8536327-4bf6-484e-b0ee-71cbf42044e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.878753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8536327-4bf6-484e-b0ee-71cbf42044e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.879044 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8536327-4bf6-484e-b0ee-71cbf42044e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:40.879710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8536327-4bf6-484e-b0ee-71cbf42044e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:40.880764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8536327-4bf6-484e-b0ee-71cbf42044e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:40.882413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8536327-4bf6-484e-b0ee-71cbf42044e4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:40.882638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa38b7-56a0-4cb9-81e0-5363f9773a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:40.883385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa38b7-56a0-4cb9-81e0-5363f9773a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.883761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa38b7-56a0-4cb9-81e0-5363f9773a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.884449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa38b7-56a0-4cb9-81e0-5363f9773a07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:40.885319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa38b7-56a0-4cb9-81e0-5363f9773a07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:40.885714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa38b7-56a0-4cb9-81e0-5363f9773a07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:40.888103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa38b7-56a0-4cb9-81e0-5363f9773a07 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:40.899106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 993/1962] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:02:40.901117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8536327-4bf6-484e-b0ee-71cbf42044e4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:40.906279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 970/1963] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:02:40.911160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa38b7-56a0-4cb9-81e0-5363f9773a07 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:40.914026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8536327-4bf6-484e-b0ee-71cbf42044e4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:40.919723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc034771-fd84-4e49-9e1c-e2352803f92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:40.920444 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc034771-fd84-4e49-9e1c-e2352803f92f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.921383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc034771-fd84-4e49-9e1c-e2352803f92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.921764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc034771-fd84-4e49-9e1c-e2352803f92f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:40.921866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa38b7-56a0-4cb9-81e0-5363f9773a07 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:40.922582 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc034771-fd84-4e49-9e1c-e2352803f92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:40.924675 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc034771-fd84-4e49-9e1c-e2352803f92f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:40.926926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c99399-9b32-44d7-8c4c-8172eff4fe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:40.927726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c99399-9b32-44d7-8c4c-8172eff4fe6a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.928009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c99399-9b32-44d7-8c4c-8172eff4fe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.928283 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c99399-9b32-44d7-8c4c-8172eff4fe6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:40.928908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c99399-9b32-44d7-8c4c-8172eff4fe6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:40.930919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c99399-9b32-44d7-8c4c-8172eff4fe6a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:40.959547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 994/1964] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:40 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:40.963471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc034771-fd84-4e49-9e1c-e2352803f92f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:40.968583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 971/1965] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:40 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:40.972386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c99399-9b32-44d7-8c4c-8172eff4fe6a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.032061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:41.064919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c99399-9b32-44d7-8c4c-8172eff4fe6a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.065248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc034771-fd84-4e49-9e1c-e2352803f92f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.071310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa74ca-1063-42e4-9bd5-087e269f81f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.071791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade3124-fcba-4ed6-8ac4-0f2920fc80c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.072097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa74ca-1063-42e4-9bd5-087e269f81f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.072385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa74ca-1063-42e4-9bd5-087e269f81f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.072478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade3124-fcba-4ed6-8ac4-0f2920fc80c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.072663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa74ca-1063-42e4-9bd5-087e269f81f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.072754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade3124-fcba-4ed6-8ac4-0f2920fc80c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.072929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade3124-fcba-4ed6-8ac4-0f2920fc80c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.073296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa74ca-1063-42e4-9bd5-087e269f81f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.073547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade3124-fcba-4ed6-8ac4-0f2920fc80c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.075271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa74ca-1063-42e4-9bd5-087e269f81f0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.075464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade3124-fcba-4ed6-8ac4-0f2920fc80c5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.097852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 972/1966] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:40 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:41.100950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 995/1967] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:40 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:41.102119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa74ca-1063-42e4-9bd5-087e269f81f0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.102255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade3124-fcba-4ed6-8ac4-0f2920fc80c5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.112567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade3124-fcba-4ed6-8ac4-0f2920fc80c5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.113070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa74ca-1063-42e4-9bd5-087e269f81f0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.117354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caeff9-8e0d-4530-86f2-9b1fd1c674a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.117727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1038b-d584-44c9-9a2c-89d9c89eac95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.117941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caeff9-8e0d-4530-86f2-9b1fd1c674a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.118217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caeff9-8e0d-4530-86f2-9b1fd1c674a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.118322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1038b-d584-44c9-9a2c-89d9c89eac95 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.118485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caeff9-8e0d-4530-86f2-9b1fd1c674a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.118594 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1038b-d584-44c9-9a2c-89d9c89eac95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.118872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1038b-d584-44c9-9a2c-89d9c89eac95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.119160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38caeff9-8e0d-4530-86f2-9b1fd1c674a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.119540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1038b-d584-44c9-9a2c-89d9c89eac95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.121100 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38caeff9-8e0d-4530-86f2-9b1fd1c674a0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.121204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1038b-d584-44c9-9a2c-89d9c89eac95 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.132803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 973/1968] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:41.136337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 996/1969] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:41.136723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1038b-d584-44c9-9a2c-89d9c89eac95 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.139993 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caeff9-8e0d-4530-86f2-9b1fd1c674a0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.146573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1038b-d584-44c9-9a2c-89d9c89eac95 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.149301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caeff9-8e0d-4530-86f2-9b1fd1c674a0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.151265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c0fbe-de2a-4eba-a4f1-8b750938267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.151860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0fbe-de2a-4eba-a4f1-8b750938267d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.152146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0fbe-de2a-4eba-a4f1-8b750938267d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.152422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0fbe-de2a-4eba-a4f1-8b750938267d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.153187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c0fbe-de2a-4eba-a4f1-8b750938267d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.154350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c07a6-eb79-40f3-a731-a982b5c87604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.154799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c0fbe-de2a-4eba-a4f1-8b750938267d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.154948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c07a6-eb79-40f3-a731-a982b5c87604 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.155259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c07a6-eb79-40f3-a731-a982b5c87604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.155529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c07a6-eb79-40f3-a731-a982b5c87604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.156147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c07a6-eb79-40f3-a731-a982b5c87604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.156268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:41.157802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c07a6-eb79-40f3-a731-a982b5c87604 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.166653 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 974/1970] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:41.171357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c0fbe-de2a-4eba-a4f1-8b750938267d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.173938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 997/1971] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:41.177726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c07a6-eb79-40f3-a731-a982b5c87604 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.184550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c0fbe-de2a-4eba-a4f1-8b750938267d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.188135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c07a6-eb79-40f3-a731-a982b5c87604 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.190045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4d5a5-2453-423d-826f-2037520b4b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.190710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d5a5-2453-423d-826f-2037520b4b4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.191021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d5a5-2453-423d-826f-2037520b4b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.191337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d5a5-2453-423d-826f-2037520b4b4f admin admin] PATH_INFO: `/v3/projects/dfe9702c4ead416b866744e75fdcd5a2/users/d58bff22d32a4ce8b6ba41d446a38c77/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.192900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9f280-3804-4602-a793-2abb397aba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.193534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9f280-3804-4602-a793-2abb397aba6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.193811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9f280-3804-4602-a793-2abb397aba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.194078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9f280-3804-4602-a793-2abb397aba6c admin admin] PATH_INFO: `/v3/projects/d64f9d1b73c24a21a4c49565d436836e/users/e2994c9e4bc24406aec8824b81f6ea0c/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.206678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4d5a5-2453-423d-826f-2037520b4b4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfe9702c4ead416b866744e75fdcd5a2, user_id=d58bff22d32a4ce8b6ba41d446a38c77, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.209429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9f280-3804-4602-a793-2abb397aba6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64f9d1b73c24a21a4c49565d436836e, user_id=e2994c9e4bc24406aec8824b81f6ea0c, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.209544 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4d5a5-2453-423d-826f-2037520b4b4f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.212156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9f280-3804-4602-a793-2abb397aba6c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.224296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 975/1972] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:41 2022] PUT /identity/v3/projects/dfe9702c4ead416b866744e75fdcd5a2/users/d58bff22d32a4ce8b6ba41d446a38c77/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:41.225506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 998/1973] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:41 2022] PUT /identity/v3/projects/d64f9d1b73c24a21a4c49565d436836e/users/e2994c9e4bc24406aec8824b81f6ea0c/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:41.232550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3ba3c-a314-40bc-8767-da64fc4613c9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.232839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3ba3c-a314-40bc-8767-da64fc4613c9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.233674 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3ba3c-a314-40bc-8767-da64fc4613c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.235665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ffcc-6c3b-4554-981a-b21f2fa1c172 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.236007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ffcc-6c3b-4554-981a-b21f2fa1c172 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.236284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ffcc-6c3b-4554-981a-b21f2fa1c172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.253103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-30d3ba3c-a314-40bc-8767-da64fc4613c9 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:41.254233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-30d3ba3c-a314-40bc-8767-da64fc4613c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:41.264940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0a17ffcc-6c3b-4554-981a-b21f2fa1c172 None None] MFA Rules not processed for user `d58bff22d32a4ce8b6ba41d446a38c77`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:41.266223 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0a17ffcc-6c3b-4554-981a-b21f2fa1c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:41.314728 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 976/1974] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:41.325827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3ba3c-a314-40bc-8767-da64fc4613c9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.327355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-30d3ba3c-a314-40bc-8767-da64fc4613c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:41.332713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 999/1975] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:41.337205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17ffcc-6c3b-4554-981a-b21f2fa1c172 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.349188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17ffcc-6c3b-4554-981a-b21f2fa1c172 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.354705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91ed72-3491-485e-94ed-6a8deef0e4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.355365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91ed72-3491-485e-94ed-6a8deef0e4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.355651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91ed72-3491-485e-94ed-6a8deef0e4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.356409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91ed72-3491-485e-94ed-6a8deef0e4b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.357056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91ed72-3491-485e-94ed-6a8deef0e4b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.361853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91ed72-3491-485e-94ed-6a8deef0e4b0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.402326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1000/1976] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:41 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:41.403623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91ed72-3491-485e-94ed-6a8deef0e4b0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.413444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3ba3c-a314-40bc-8767-da64fc4613c9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.462197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:41.469824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba987e08-fd3e-45cd-8454-49641a58b592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.470630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba987e08-fd3e-45cd-8454-49641a58b592 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.471369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba987e08-fd3e-45cd-8454-49641a58b592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.471878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba987e08-fd3e-45cd-8454-49641a58b592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.479461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91ed72-3491-485e-94ed-6a8deef0e4b0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.484224 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba987e08-fd3e-45cd-8454-49641a58b592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.484897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38614d15-04d8-48a4-adb0-8dfac454cffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.485506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38614d15-04d8-48a4-adb0-8dfac454cffa admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.485830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38614d15-04d8-48a4-adb0-8dfac454cffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.486101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38614d15-04d8-48a4-adb0-8dfac454cffa admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.486264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba987e08-fd3e-45cd-8454-49641a58b592 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.486925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38614d15-04d8-48a4-adb0-8dfac454cffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.488912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38614d15-04d8-48a4-adb0-8dfac454cffa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.496421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 977/1977] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:02:41.516688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1001/1978] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:41 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:41.518501 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba987e08-fd3e-45cd-8454-49641a58b592 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.532312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba987e08-fd3e-45cd-8454-49641a58b592 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.537294 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3325b3-4424-41f2-bc98-f7728e3d1eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.537966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3325b3-4424-41f2-bc98-f7728e3d1eab admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.538307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3325b3-4424-41f2-bc98-f7728e3d1eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.538649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3325b3-4424-41f2-bc98-f7728e3d1eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.539378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3325b3-4424-41f2-bc98-f7728e3d1eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.541059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3325b3-4424-41f2-bc98-f7728e3d1eab admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.555507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 978/1979] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:41.557063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38614d15-04d8-48a4-adb0-8dfac454cffa admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.569824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38614d15-04d8-48a4-adb0-8dfac454cffa admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.575530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf12dff-4c3d-419a-8d25-4730950d70fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.576276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf12dff-4c3d-419a-8d25-4730950d70fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.576664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf12dff-4c3d-419a-8d25-4730950d70fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.577052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf12dff-4c3d-419a-8d25-4730950d70fb admin admin] PATH_INFO: `/v3/projects/180f85d1db2f42aa86628fa976759e7e/users/5e5168d9d4574b7db034d8b21950a91d/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.595635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf12dff-4c3d-419a-8d25-4730950d70fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=180f85d1db2f42aa86628fa976759e7e, user_id=5e5168d9d4574b7db034d8b21950a91d, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.599560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf12dff-4c3d-419a-8d25-4730950d70fb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.623471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1002/1980] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:41 2022] PUT /identity/v3/projects/180f85d1db2f42aa86628fa976759e7e/users/5e5168d9d4574b7db034d8b21950a91d/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:41.626085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3325b3-4424-41f2-bc98-f7728e3d1eab admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.642871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3325b3-4424-41f2-bc98-f7728e3d1eab admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.696477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:41.785684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b618c-a900-410a-ac47-e9439d55571b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.786411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b618c-a900-410a-ac47-e9439d55571b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.786938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b618c-a900-410a-ac47-e9439d55571b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.794426 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b618c-a900-410a-ac47-e9439d55571b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.796831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b618c-a900-410a-ac47-e9439d55571b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.799155 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b618c-a900-410a-ac47-e9439d55571b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.817580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 979/1981] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:41.820299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf12dff-4c3d-419a-8d25-4730950d70fb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.842277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf12dff-4c3d-419a-8d25-4730950d70fb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.876858 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbdf11-9476-410e-a4f2-ab86d77dd287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.880585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbdf11-9476-410e-a4f2-ab86d77dd287 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.881113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbdf11-9476-410e-a4f2-ab86d77dd287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.881621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbdf11-9476-410e-a4f2-ab86d77dd287 admin admin] PATH_INFO: `/v3/projects/180f85d1db2f42aa86628fa976759e7e/users/5e5168d9d4574b7db034d8b21950a91d/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.896469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbdf11-9476-410e-a4f2-ab86d77dd287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=180f85d1db2f42aa86628fa976759e7e, user_id=5e5168d9d4574b7db034d8b21950a91d, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.903073 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbdf11-9476-410e-a4f2-ab86d77dd287 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.945418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1003/1982] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:41 2022] PUT /identity/v3/projects/180f85d1db2f42aa86628fa976759e7e/users/5e5168d9d4574b7db034d8b21950a91d/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:41.949117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47314f-7298-4c67-bc53-21415d783b44 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.949457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47314f-7298-4c67-bc53-21415d783b44 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.949744 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47314f-7298-4c67-bc53-21415d783b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.993562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2b47314f-7298-4c67-bc53-21415d783b44 None None] MFA Rules not processed for user `5e5168d9d4574b7db034d8b21950a91d`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:41.995542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2b47314f-7298-4c67-bc53-21415d783b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:42.133857 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 980/1983] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:42.152743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbdf11-9476-410e-a4f2-ab86d77dd287 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:42.154924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c4fbdf11-9476-410e-a4f2-ab86d77dd287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:42.274340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbdf11-9476-410e-a4f2-ab86d77dd287 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:42.297221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fea39b-4a8d-482c-b8bc-9f223381120c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:42.297854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fea39b-4a8d-482c-b8bc-9f223381120c service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:42.298132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fea39b-4a8d-482c-b8bc-9f223381120c service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:42.298403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fea39b-4a8d-482c-b8bc-9f223381120c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:42.316598 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fea39b-4a8d-482c-b8bc-9f223381120c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:42.319577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fea39b-4a8d-482c-b8bc-9f223381120c service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:42.351092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1004/1984] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:42.451878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15c8c4-c22b-48f7-ada7-6ae1825c0028 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:42.451878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15c8c4-c22b-48f7-ada7-6ae1825c0028 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:42.452620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15c8c4-c22b-48f7-ada7-6ae1825c0028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:42.490649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ed15c8c4-c22b-48f7-ada7-6ae1825c0028 None None] MFA Rules not processed for user `e2994c9e4bc24406aec8824b81f6ea0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:42.492282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ed15c8c4-c22b-48f7-ada7-6ae1825c0028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:42.581779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 981/1985] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:42.586403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fea39b-4a8d-482c-b8bc-9f223381120c service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:42.593958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15c8c4-c22b-48f7-ada7-6ae1825c0028 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:42.640961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:42.683363 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15c8c4-c22b-48f7-ada7-6ae1825c0028 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:42.696061 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1b477-20b0-4af4-886e-eb190b706b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:42.696252 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fea39b-4a8d-482c-b8bc-9f223381120c service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:42.696946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1b477-20b0-4af4-886e-eb190b706b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:42.697380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1b477-20b0-4af4-886e-eb190b706b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:42.697993 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1b477-20b0-4af4-886e-eb190b706b5e admin admin] PATH_INFO: `/v3/users/d58bff22d32a4ce8b6ba41d446a38c77` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:42.699482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1b477-20b0-4af4-886e-eb190b706b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d58bff22d32a4ce8b6ba41d446a38c77)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:42.701524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1b477-20b0-4af4-886e-eb190b706b5e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:42.702755 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d4ea6-28f8-427e-af59-30ac2cb1d6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:42.703433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d4ea6-28f8-427e-af59-30ac2cb1d6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:42.703723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d4ea6-28f8-427e-af59-30ac2cb1d6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:42.703988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d4ea6-28f8-427e-af59-30ac2cb1d6d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:42.704652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d4ea6-28f8-427e-af59-30ac2cb1d6d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:42.706847 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d4ea6-28f8-427e-af59-30ac2cb1d6d4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:42.757044 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1005/1986] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:42 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:42.758219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d4ea6-28f8-427e-af59-30ac2cb1d6d4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:42.779525 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3b1b477-20b0-4af4-886e-eb190b706b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd58bff22d32a4ce8b6ba41d446a38c77'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:42.792660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:42.804927 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3b1b477-20b0-4af4-886e-eb190b706b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd58bff22d32a4ce8b6ba41d446a38c77'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:42.805835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3b1b477-20b0-4af4-886e-eb190b706b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd58bff22d32a4ce8b6ba41d446a38c77'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:42.806540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3b1b477-20b0-4af4-886e-eb190b706b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd58bff22d32a4ce8b6ba41d446a38c77'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:42.822637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3b1b477-20b0-4af4-886e-eb190b706b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd58bff22d32a4ce8b6ba41d446a38c77'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:42.838802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d4ea6-28f8-427e-af59-30ac2cb1d6d4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:42.847481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ff67c-78a1-4712-bdd4-1330aa8a360b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:42.848019 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ff67c-78a1-4712-bdd4-1330aa8a360b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:42.848402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 982/1987] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:42 2022] DELETE /identity/v3/users/d58bff22d32a4ce8b6ba41d446a38c77 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:42.848516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ff67c-78a1-4712-bdd4-1330aa8a360b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:42.852506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ff67c-78a1-4712-bdd4-1330aa8a360b admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:42.853185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ff67c-78a1-4712-bdd4-1330aa8a360b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:42.853849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f29c7-55c9-4a5b-81a6-45264608c12d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:42.854162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f29c7-55c9-4a5b-81a6-45264608c12d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:42.854406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f29c7-55c9-4a5b-81a6-45264608c12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:42.859605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ff67c-78a1-4712-bdd4-1330aa8a360b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:42.881650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-977f29c7-55c9-4a5b-81a6-45264608c12d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:42.883569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-977f29c7-55c9-4a5b-81a6-45264608c12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:42.891509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1006/1988] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:42 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:42.895347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ff67c-78a1-4712-bdd4-1330aa8a360b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:42.897099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-353ff67c-78a1-4712-bdd4-1330aa8a360b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:42.910254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ff67c-78a1-4712-bdd4-1330aa8a360b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:42.966279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 983/1989] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:42.977900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f29c7-55c9-4a5b-81a6-45264608c12d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:42.979868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-977f29c7-55c9-4a5b-81a6-45264608c12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:42.991648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d097d6-2941-4c53-9fc0-d4e6439583c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:42.992270 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d097d6-2941-4c53-9fc0-d4e6439583c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:42.992598 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d097d6-2941-4c53-9fc0-d4e6439583c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:42.992866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d097d6-2941-4c53-9fc0-d4e6439583c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:42.993559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d097d6-2941-4c53-9fc0-d4e6439583c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:42.995186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d097d6-2941-4c53-9fc0-d4e6439583c8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:43.009877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1007/1990] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:02:43.010871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d097d6-2941-4c53-9fc0-d4e6439583c8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:43.021092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d097d6-2941-4c53-9fc0-d4e6439583c8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:43.025634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a66b1f-65bb-402f-82d5-e292523bb2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:43.026267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a66b1f-65bb-402f-82d5-e292523bb2e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:43.026545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a66b1f-65bb-402f-82d5-e292523bb2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:43.026818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a66b1f-65bb-402f-82d5-e292523bb2e7 admin admin] PATH_INFO: `/v3/projects/4a32cd6bf5264316bb70d9ae94e87876/users/3b5137ff267e4566ab310723c4757014/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:43.042584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f29c7-55c9-4a5b-81a6-45264608c12d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:43.043330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a66b1f-65bb-402f-82d5-e292523bb2e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a32cd6bf5264316bb70d9ae94e87876, user_id=3b5137ff267e4566ab310723c4757014, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:43.046138 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a66b1f-65bb-402f-82d5-e292523bb2e7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:43.046254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18422cbc-38bf-4217-a7ee-42fe0e84a0f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:43.046819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18422cbc-38bf-4217-a7ee-42fe0e84a0f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:43.047130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18422cbc-38bf-4217-a7ee-42fe0e84a0f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:43.047406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18422cbc-38bf-4217-a7ee-42fe0e84a0f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:43.054044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:43.057725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18422cbc-38bf-4217-a7ee-42fe0e84a0f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:43.059504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18422cbc-38bf-4217-a7ee-42fe0e84a0f5 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:43.060917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1008/1991] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:43 2022] PUT /identity/v3/projects/4a32cd6bf5264316bb70d9ae94e87876/users/3b5137ff267e4566ab310723c4757014/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:43.065493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a66b1f-65bb-402f-82d5-e292523bb2e7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:43.072150 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 984/1992] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:02:43.210761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a66b1f-65bb-402f-82d5-e292523bb2e7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:43.216971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4cc2d-5ae0-45c6-a2da-e2b2f6e35189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:43.218182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4cc2d-5ae0-45c6-a2da-e2b2f6e35189 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:43.218725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4cc2d-5ae0-45c6-a2da-e2b2f6e35189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:43.219233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4cc2d-5ae0-45c6-a2da-e2b2f6e35189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:43.220143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4cc2d-5ae0-45c6-a2da-e2b2f6e35189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:43.221985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4cc2d-5ae0-45c6-a2da-e2b2f6e35189 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:43.235317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1009/1993] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:43.240254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18422cbc-38bf-4217-a7ee-42fe0e84a0f5 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:43.253241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18422cbc-38bf-4217-a7ee-42fe0e84a0f5 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:43.260688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a857de-44a2-4131-94a0-9f510b9b3af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:43.261385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a857de-44a2-4131-94a0-9f510b9b3af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:43.262192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a857de-44a2-4131-94a0-9f510b9b3af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:43.262478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a857de-44a2-4131-94a0-9f510b9b3af0 admin admin] PATH_INFO: `/v3/projects/4a32cd6bf5264316bb70d9ae94e87876/users/3b5137ff267e4566ab310723c4757014/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:43.265525 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a857de-44a2-4131-94a0-9f510b9b3af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a32cd6bf5264316bb70d9ae94e87876, user_id=3b5137ff267e4566ab310723c4757014, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:43.269585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a857de-44a2-4131-94a0-9f510b9b3af0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:43.284957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 985/1994] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:43 2022] PUT /identity/v3/projects/4a32cd6bf5264316bb70d9ae94e87876/users/3b5137ff267e4566ab310723c4757014/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:43.520594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4cc2d-5ae0-45c6-a2da-e2b2f6e35189 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:43.522791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f1d4cc2d-5ae0-45c6-a2da-e2b2f6e35189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:43.527659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a857de-44a2-4131-94a0-9f510b9b3af0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:43.536315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a5a857de-44a2-4131-94a0-9f510b9b3af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:43.542380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4cc2d-5ae0-45c6-a2da-e2b2f6e35189 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:43.618316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:43.669731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:43.671551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:43.671966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:43.672409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] PATH_INFO: `/v3/projects/dfe9702c4ead416b866744e75fdcd5a2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:43.673796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfe9702c4ead416b866744e75fdcd5a2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:43.675956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:43.704819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a857de-44a2-4131-94a0-9f510b9b3af0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:43.710419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d11185-8d50-4367-9fc3-b62693f681d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:43.711112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d11185-8d50-4367-9fc3-b62693f681d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:43.711400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d11185-8d50-4367-9fc3-b62693f681d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:43.711681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d11185-8d50-4367-9fc3-b62693f681d7 admin admin] PATH_INFO: `/v3/users/217b8c643ed94ec38304d3fcc8519fbc` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:43.712900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d11185-8d50-4367-9fc3-b62693f681d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=217b8c643ed94ec38304d3fcc8519fbc)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:43.715071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d11185-8d50-4367-9fc3-b62693f681d7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:43.781568 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfe9702c4ead416b866744e75fdcd5a2', 'request_id': 'req-ab9ccafd-3770-4b85-a108-b80d5ce91e17'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:43.788975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-53d11185-8d50-4367-9fc3-b62693f681d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '217b8c643ed94ec38304d3fcc8519fbc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:43.793945 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfe9702c4ead416b866744e75fdcd5a2', 'request_id': 'req-ab9ccafd-3770-4b85-a108-b80d5ce91e17'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:43.805731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfe9702c4ead416b866744e75fdcd5a2', 'request_id': 'req-ab9ccafd-3770-4b85-a108-b80d5ce91e17'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:43.832786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-53d11185-8d50-4367-9fc3-b62693f681d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '217b8c643ed94ec38304d3fcc8519fbc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:43.833974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-53d11185-8d50-4367-9fc3-b62693f681d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '217b8c643ed94ec38304d3fcc8519fbc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:43.834938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-53d11185-8d50-4367-9fc3-b62693f681d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '217b8c643ed94ec38304d3fcc8519fbc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:43.841071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] The token cache is being invalidate because project dfe9702c4ead416b866744e75fdcd5a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:43.843225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:43.844010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:43.846766 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1010/1995] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:43 2022] DELETE /identity/v3/projects/dfe9702c4ead416b866744e75fdcd5a2 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:43.850611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:43.851609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-53d11185-8d50-4367-9fc3-b62693f681d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '217b8c643ed94ec38304d3fcc8519fbc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:43.852388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:43.872380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 986/1996] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:43 2022] DELETE /identity/v3/users/217b8c643ed94ec38304d3fcc8519fbc => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:43.877844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032c056-93ae-43f9-95bd-1cf908dd7d60 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:43.878416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032c056-93ae-43f9-95bd-1cf908dd7d60 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:43.878873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032c056-93ae-43f9-95bd-1cf908dd7d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:43.907548 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4032c056-93ae-43f9-95bd-1cf908dd7d60 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:43.909119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4032c056-93ae-43f9-95bd-1cf908dd7d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:43.957414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ccafd-3770-4b85-a108-b80d5ce91e17 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:43.985376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:44.002811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 987/1997] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:44.010865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032c056-93ae-43f9-95bd-1cf908dd7d60 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:44.012601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4032c056-93ae-43f9-95bd-1cf908dd7d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:44.044523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:44.045236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:44.045559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:44.045941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] PATH_INFO: `/v3/users/5e5168d9d4574b7db034d8b21950a91d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:44.047660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e5168d9d4574b7db034d8b21950a91d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:44.049687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:44.052588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:44.103599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032c056-93ae-43f9-95bd-1cf908dd7d60 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:44.112878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eedabb-7d38-41bc-b0c8-0edcf46453f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:44.114078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eedabb-7d38-41bc-b0c8-0edcf46453f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:44.115217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eedabb-7d38-41bc-b0c8-0edcf46453f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:44.116728 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eedabb-7d38-41bc-b0c8-0edcf46453f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:44.133669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e5168d9d4574b7db034d8b21950a91d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:44.134970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e5168d9d4574b7db034d8b21950a91d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:44.140355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eedabb-7d38-41bc-b0c8-0edcf46453f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:44.142168 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eedabb-7d38-41bc-b0c8-0edcf46453f9 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:44.151245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e5168d9d4574b7db034d8b21950a91d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:44.153574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 988/1998] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:02:44.160698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e5168d9d4574b7db034d8b21950a91d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:44.168564 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e5168d9d4574b7db034d8b21950a91d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:44.175758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1011/1999] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:02:43 2022] DELETE /identity/v3/users/5e5168d9d4574b7db034d8b21950a91d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:44.657757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a0072-a310-4fa8-810b-babb9b6f932a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:44.658099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a0072-a310-4fa8-810b-babb9b6f932a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:44.658383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a0072-a310-4fa8-810b-babb9b6f932a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:44.704423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a40a0072-a310-4fa8-810b-babb9b6f932a None None] MFA Rules not processed for user `3b5137ff267e4566ab310723c4757014`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:44.706523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a40a0072-a310-4fa8-810b-babb9b6f932a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:44.778061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:44.781300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:44.919329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254e345-1f14-4baa-9aa8-79923fa75b37 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:44.921352 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 989/2000] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:44.922432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a0072-a310-4fa8-810b-babb9b6f932a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:44.924502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a40a0072-a310-4fa8-810b-babb9b6f932a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:44.935033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:44.936694 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:44.936997 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:44.937278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] PATH_INFO: `/v3/projects/180f85d1db2f42aa86628fa976759e7e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:44.940796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a0072-a310-4fa8-810b-babb9b6f932a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:44.943318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=180f85d1db2f42aa86628fa976759e7e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:44.945264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:44.947099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:44.948418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:44.948418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:44.948418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] PATH_INFO: `/v3/projects/b9e7b8f5afff4a518c2252d29001ebaf` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:44.950316 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e7b8f5afff4a518c2252d29001ebaf)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:44.953696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.028763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e7b8f5afff4a518c2252d29001ebaf', 'request_id': 'req-655857c5-c5f7-484d-8fc8-5e40a0961d76'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:45.033385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '180f85d1db2f42aa86628fa976759e7e', 'request_id': 'req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:45.040981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e7b8f5afff4a518c2252d29001ebaf', 'request_id': 'req-655857c5-c5f7-484d-8fc8-5e40a0961d76'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:45.048357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '180f85d1db2f42aa86628fa976759e7e', 'request_id': 'req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:45.050203 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e7b8f5afff4a518c2252d29001ebaf', 'request_id': 'req-655857c5-c5f7-484d-8fc8-5e40a0961d76'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:45.059517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '180f85d1db2f42aa86628fa976759e7e', 'request_id': 'req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:45.082041 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] The token cache is being invalidate because project b9e7b8f5afff4a518c2252d29001ebaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:45.082353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:45.083173 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:45.084851 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 990/2001] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:44 2022] DELETE /identity/v3/projects/b9e7b8f5afff4a518c2252d29001ebaf => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:45.087970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.092924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:45.095578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] The token cache is being invalidate because project 180f85d1db2f42aa86628fa976759e7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:02:45.095903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:45.096648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e0e8e62-e30a-4958-8ff9-b56e42b66e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:02:45.104180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1012/2002] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:02:44 2022] DELETE /identity/v3/projects/180f85d1db2f42aa86628fa976759e7e => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:45.109989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64459d3c-9482-4f46-83d8-5d31845e009d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.110287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64459d3c-9482-4f46-83d8-5d31845e009d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.112231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64459d3c-9482-4f46-83d8-5d31845e009d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.122170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:45.159977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-64459d3c-9482-4f46-83d8-5d31845e009d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:45.161220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-64459d3c-9482-4f46-83d8-5d31845e009d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:45.180446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:45.193352 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655857c5-c5f7-484d-8fc8-5e40a0961d76 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.199778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4da85-707d-430d-9984-2117f1122def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.200713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4da85-707d-430d-9984-2117f1122def service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.201152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4da85-707d-430d-9984-2117f1122def service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.201728 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4da85-707d-430d-9984-2117f1122def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.205691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ebe4da85-707d-430d-9984-2117f1122def service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:45.306204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4da85-707d-430d-9984-2117f1122def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.307704 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4da85-707d-430d-9984-2117f1122def service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.311958 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1013/2003] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:45.315741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a3ded-de91-48ff-a8e4-c5bbc494f986 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.316854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a3ded-de91-48ff-a8e4-c5bbc494f986 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.317401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a3ded-de91-48ff-a8e4-c5bbc494f986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.324415 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 991/2004] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:45.325683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d18693-78db-4f3c-9e55-3e4b19fa79ed None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.325980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d18693-78db-4f3c-9e55-3e4b19fa79ed None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.326260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d18693-78db-4f3c-9e55-3e4b19fa79ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.354109 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-52d18693-78db-4f3c-9e55-3e4b19fa79ed None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:45.355741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-52d18693-78db-4f3c-9e55-3e4b19fa79ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:45.374283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-eb9a3ded-de91-48ff-a8e4-c5bbc494f986 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:45.375512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eb9a3ded-de91-48ff-a8e4-c5bbc494f986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:45.378873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 992/2005] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:45.380128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d18693-78db-4f3c-9e55-3e4b19fa79ed None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.395663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d18693-78db-4f3c-9e55-3e4b19fa79ed None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.401648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1014/2006] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:45.402874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236ceb4-4303-46d9-a5a1-e27e961e7bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.403892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236ceb4-4303-46d9-a5a1-e27e961e7bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.404391 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236ceb4-4303-46d9-a5a1-e27e961e7bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.404924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236ceb4-4303-46d9-a5a1-e27e961e7bc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.405866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236ceb4-4303-46d9-a5a1-e27e961e7bc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:45.406401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236ceb4-4303-46d9-a5a1-e27e961e7bc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.408830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236ceb4-4303-46d9-a5a1-e27e961e7bc0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.410247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6855a9-7240-4796-844b-cd66c6e3d0af None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.411493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6855a9-7240-4796-844b-cd66c6e3d0af None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.411785 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6855a9-7240-4796-844b-cd66c6e3d0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.431816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 993/2007] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:45.436752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236ceb4-4303-46d9-a5a1-e27e961e7bc0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.448084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1f6855a9-7240-4796-844b-cd66c6e3d0af None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:45.449390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1f6855a9-7240-4796-844b-cd66c6e3d0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:45.450482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236ceb4-4303-46d9-a5a1-e27e961e7bc0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.457137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea449-7668-4d94-9f8c-1df4ce830cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.458092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea449-7668-4d94-9f8c-1df4ce830cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.458526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea449-7668-4d94-9f8c-1df4ce830cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.458949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea449-7668-4d94-9f8c-1df4ce830cf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.459499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1015/2008] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:45.460292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dea449-7668-4d94-9f8c-1df4ce830cf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.463468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dea449-7668-4d94-9f8c-1df4ce830cf4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.463994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6855a9-7240-4796-844b-cd66c6e3d0af None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.475246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6855a9-7240-4796-844b-cd66c6e3d0af None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.480708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b1eea-54de-49d0-b575-7eeb52ba4c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.481323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b1eea-54de-49d0-b575-7eeb52ba4c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.481603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b1eea-54de-49d0-b575-7eeb52ba4c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.481875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b1eea-54de-49d0-b575-7eeb52ba4c74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.483620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b1eea-54de-49d0-b575-7eeb52ba4c74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:02:45.484110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b1eea-54de-49d0-b575-7eeb52ba4c74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.486401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b1eea-54de-49d0-b575-7eeb52ba4c74 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.501541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1016/2009] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:02:45.505566 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 994/2010] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:45 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:45.507766 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b1eea-54de-49d0-b575-7eeb52ba4c74 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.509216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea449-7668-4d94-9f8c-1df4ce830cf4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.518801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b1eea-54de-49d0-b575-7eeb52ba4c74 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.582423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea449-7668-4d94-9f8c-1df4ce830cf4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.587631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307250f2-6fa1-4d9b-820a-81db644beb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.588249 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307250f2-6fa1-4d9b-820a-81db644beb44 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.588539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307250f2-6fa1-4d9b-820a-81db644beb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.588815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307250f2-6fa1-4d9b-820a-81db644beb44 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.589451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307250f2-6fa1-4d9b-820a-81db644beb44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.591448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307250f2-6fa1-4d9b-820a-81db644beb44 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.604903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5d17-78d4-450d-a557-95a7dbd248d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.605656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5d17-78d4-450d-a557-95a7dbd248d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.605961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5d17-78d4-450d-a557-95a7dbd248d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.606418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5d17-78d4-450d-a557-95a7dbd248d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.607354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5d17-78d4-450d-a557-95a7dbd248d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.609799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5d17-78d4-450d-a557-95a7dbd248d3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.617992 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 995/2011] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:45 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:45.619109 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307250f2-6fa1-4d9b-820a-81db644beb44 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.634825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307250f2-6fa1-4d9b-820a-81db644beb44 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.641257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1017/2012] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:45 2022] POST /identity/v3/projects => generated 409 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:45.644225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae6034-3968-47f7-9e34-e7bb4d7a8b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.644603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae6034-3968-47f7-9e34-e7bb4d7a8b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.645119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae6034-3968-47f7-9e34-e7bb4d7a8b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.645454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae6034-3968-47f7-9e34-e7bb4d7a8b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.646279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5d17-78d4-450d-a557-95a7dbd248d3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.646786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae6034-3968-47f7-9e34-e7bb4d7a8b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.648540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae6034-3968-47f7-9e34-e7bb4d7a8b7d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.660789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 996/2013] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:45.666116 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae6034-3968-47f7-9e34-e7bb4d7a8b7d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.677409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:45.691120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae6034-3968-47f7-9e34-e7bb4d7a8b7d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.771913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5d17-78d4-450d-a557-95a7dbd248d3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.778420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8fc89-e3f3-4706-8250-bfd655456162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.779687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8fc89-e3f3-4706-8250-bfd655456162 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.780369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8fc89-e3f3-4706-8250-bfd655456162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.781097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8fc89-e3f3-4706-8250-bfd655456162 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.782982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8fc89-e3f3-4706-8250-bfd655456162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.787109 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8fc89-e3f3-4706-8250-bfd655456162 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.812850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:45.820256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3ac3a-95ae-4872-81a4-e42e593f20df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.820890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3ac3a-95ae-4872-81a4-e42e593f20df admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.821241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3ac3a-95ae-4872-81a4-e42e593f20df admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.821439 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3ac3a-95ae-4872-81a4-e42e593f20df admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.824423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3ac3a-95ae-4872-81a4-e42e593f20df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.824799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3ac3a-95ae-4872-81a4-e42e593f20df admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.825432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1018/2014] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:45 2022] POST /identity/v3/users => generated 305 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:45.831457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8fc89-e3f3-4706-8250-bfd655456162 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.851704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8fc89-e3f3-4706-8250-bfd655456162 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.860113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac9d22-dd37-4949-94c7-eb54ff75f75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.860878 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac9d22-dd37-4949-94c7-eb54ff75f75a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.861163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac9d22-dd37-4949-94c7-eb54ff75f75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.861501 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac9d22-dd37-4949-94c7-eb54ff75f75a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.863223 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac9d22-dd37-4949-94c7-eb54ff75f75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.865120 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac9d22-dd37-4949-94c7-eb54ff75f75a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.872238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 997/2015] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:45.877557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3ac3a-95ae-4872-81a4-e42e593f20df admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.882568 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1019/2016] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:45.883904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac9d22-dd37-4949-94c7-eb54ff75f75a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.896125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac9d22-dd37-4949-94c7-eb54ff75f75a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.900592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3ac3a-95ae-4872-81a4-e42e593f20df admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.901886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3a1c6-41df-48b6-bbf0-91ba8cde45cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.902562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3a1c6-41df-48b6-bbf0-91ba8cde45cb admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.902849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3a1c6-41df-48b6-bbf0-91ba8cde45cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.903169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3a1c6-41df-48b6-bbf0-91ba8cde45cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.903922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3a1c6-41df-48b6-bbf0-91ba8cde45cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.905915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3a1c6-41df-48b6-bbf0-91ba8cde45cb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.916374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e8517-417e-454a-9b26-7fa1e8c23742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.917795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8517-417e-454a-9b26-7fa1e8c23742 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.922302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1020/2017] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:45.922820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8517-417e-454a-9b26-7fa1e8c23742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.923318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8517-417e-454a-9b26-7fa1e8c23742 admin admin] PATH_INFO: `/v3/projects/cbf40ccacdc74575909f541214adf6f7/users/41227378a57c43219e4ae460c642f834/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.923668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3a1c6-41df-48b6-bbf0-91ba8cde45cb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:45.937455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3a1c6-41df-48b6-bbf0-91ba8cde45cb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:45.942925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333d87a-4f18-4fa6-9e62-3f2223ed7127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:45.943851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333d87a-4f18-4fa6-9e62-3f2223ed7127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.944129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333d87a-4f18-4fa6-9e62-3f2223ed7127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.944401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333d87a-4f18-4fa6-9e62-3f2223ed7127 admin admin] PATH_INFO: `/v3/projects/06d10e93c76a44ed8e741aa6cdf6457f/users/3a6ba03e851d4d778b2a63bfbf440616/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.948043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e8517-417e-454a-9b26-7fa1e8c23742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf40ccacdc74575909f541214adf6f7, user_id=41227378a57c43219e4ae460c642f834, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.950938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e8517-417e-454a-9b26-7fa1e8c23742 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.961780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333d87a-4f18-4fa6-9e62-3f2223ed7127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d10e93c76a44ed8e741aa6cdf6457f, user_id=3a6ba03e851d4d778b2a63bfbf440616, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:45.964894 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333d87a-4f18-4fa6-9e62-3f2223ed7127 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:45.969382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 998/2018] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:45 2022] PUT /identity/v3/projects/cbf40ccacdc74575909f541214adf6f7/users/41227378a57c43219e4ae460c642f834/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:45.975344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72351b9-cf1b-4a6c-9f3c-f861487c380a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:45.976259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72351b9-cf1b-4a6c-9f3c-f861487c380a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:45.976571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72351b9-cf1b-4a6c-9f3c-f861487c380a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:45.987824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1021/2019] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:45 2022] PUT /identity/v3/projects/06d10e93c76a44ed8e741aa6cdf6457f/users/3a6ba03e851d4d778b2a63bfbf440616/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:02:45.999932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605c71c-9ccc-4f0b-a22d-11523cca5c30 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:46.000189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605c71c-9ccc-4f0b-a22d-11523cca5c30 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:46.000440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605c71c-9ccc-4f0b-a22d-11523cca5c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:46.001652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d72351b9-cf1b-4a6c-9f3c-f861487c380a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:46.003521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d72351b9-cf1b-4a6c-9f3c-f861487c380a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:46.048716 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d605c71c-9ccc-4f0b-a22d-11523cca5c30 None None] MFA Rules not processed for user `3a6ba03e851d4d778b2a63bfbf440616`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:46.050282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d605c71c-9ccc-4f0b-a22d-11523cca5c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:46.099521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 999/2020] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:46.109899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72351b9-cf1b-4a6c-9f3c-f861487c380a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:46.112082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d72351b9-cf1b-4a6c-9f3c-f861487c380a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:46.151295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1022/2021] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:45 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:46.152410 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605c71c-9ccc-4f0b-a22d-11523cca5c30 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:46.164307 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605c71c-9ccc-4f0b-a22d-11523cca5c30 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:46.169325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa981e4-1805-4355-8afe-81cf5e7cd33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:46.170046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa981e4-1805-4355-8afe-81cf5e7cd33a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:46.170445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa981e4-1805-4355-8afe-81cf5e7cd33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:46.170736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa981e4-1805-4355-8afe-81cf5e7cd33a admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:46.171475 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa981e4-1805-4355-8afe-81cf5e7cd33a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:46.173432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa981e4-1805-4355-8afe-81cf5e7cd33a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:46.180661 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72351b9-cf1b-4a6c-9f3c-f861487c380a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:46.185594 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ebdbd-cb24-4078-91a3-890521a720b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:46.186961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ebdbd-cb24-4078-91a3-890521a720b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:46.187406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ebdbd-cb24-4078-91a3-890521a720b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:46.187727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ebdbd-cb24-4078-91a3-890521a720b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:46.198933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ebdbd-cb24-4078-91a3-890521a720b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:46.200417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ebdbd-cb24-4078-91a3-890521a720b5 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:46.212209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1023/2022] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:02:46 2022] POST /identity/v3/projects => generated 407 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:02:46.212507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1000/2023] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:46.213075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa981e4-1805-4355-8afe-81cf5e7cd33a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:46.324602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa981e4-1805-4355-8afe-81cf5e7cd33a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:46.347026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5bef8-bd70-4a9c-bca7-b9e5adc0b58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:46.347694 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5bef8-bd70-4a9c-bca7-b9e5adc0b58b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:46.347974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5bef8-bd70-4a9c-bca7-b9e5adc0b58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:46.348764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5bef8-bd70-4a9c-bca7-b9e5adc0b58b admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:46.348890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5bef8-bd70-4a9c-bca7-b9e5adc0b58b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:46.350826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5bef8-bd70-4a9c-bca7-b9e5adc0b58b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:46.402070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1024/2024] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:02:46 2022] POST /identity/v3/users => generated 310 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:02:46.405361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ebdbd-cb24-4078-91a3-890521a720b5 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:46.426648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ebdbd-cb24-4078-91a3-890521a720b5 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:46.443698 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91720cfe-9ea6-45fe-bd85-d6d8cba4b662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:46.444454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91720cfe-9ea6-45fe-bd85-d6d8cba4b662 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:46.444839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91720cfe-9ea6-45fe-bd85-d6d8cba4b662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:46.445221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91720cfe-9ea6-45fe-bd85-d6d8cba4b662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:46.445961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91720cfe-9ea6-45fe-bd85-d6d8cba4b662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:46.447809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91720cfe-9ea6-45fe-bd85-d6d8cba4b662 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:46.479460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1001/2025] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:46.485993 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5bef8-bd70-4a9c-bca7-b9e5adc0b58b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:46.505066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5bef8-bd70-4a9c-bca7-b9e5adc0b58b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:46.516531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77ab22-2da7-4c44-8ea4-3f0cffdc4706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:46.517487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77ab22-2da7-4c44-8ea4-3f0cffdc4706 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:46.519493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77ab22-2da7-4c44-8ea4-3f0cffdc4706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:46.519932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77ab22-2da7-4c44-8ea4-3f0cffdc4706 admin admin] PATH_INFO: `/v3/projects/02281fed3209406795711020e53b7162/users/3d9cade97a184adcad958e659e0ffcfc/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:46.541809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:46.560015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77ab22-2da7-4c44-8ea4-3f0cffdc4706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02281fed3209406795711020e53b7162, user_id=3d9cade97a184adcad958e659e0ffcfc, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:46.571273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77ab22-2da7-4c44-8ea4-3f0cffdc4706 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:46.602983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1025/2026] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:46 2022] PUT /identity/v3/projects/02281fed3209406795711020e53b7162/users/3d9cade97a184adcad958e659e0ffcfc/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:46.607384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91720cfe-9ea6-45fe-bd85-d6d8cba4b662 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:46.638668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91720cfe-9ea6-45fe-bd85-d6d8cba4b662 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:46.645928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:46.752023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8721b36-0dfa-4ece-ade5-895ee5415e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:46.752850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8721b36-0dfa-4ece-ade5-895ee5415e3f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:46.753086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8721b36-0dfa-4ece-ade5-895ee5415e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:46.753366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8721b36-0dfa-4ece-ade5-895ee5415e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:46.754025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8721b36-0dfa-4ece-ade5-895ee5415e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:46.755803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8721b36-0dfa-4ece-ade5-895ee5415e3f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:46.768445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1002/2027] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:46.772967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77ab22-2da7-4c44-8ea4-3f0cffdc4706 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:46.786801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77ab22-2da7-4c44-8ea4-3f0cffdc4706 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:46.793883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943da2db-2e0b-4e70-ae8f-ec0356489278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:46.797145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943da2db-2e0b-4e70-ae8f-ec0356489278 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:46.797626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943da2db-2e0b-4e70-ae8f-ec0356489278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:46.798071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943da2db-2e0b-4e70-ae8f-ec0356489278 admin admin] PATH_INFO: `/v3/projects/02281fed3209406795711020e53b7162/users/3d9cade97a184adcad958e659e0ffcfc/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:46.801746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943da2db-2e0b-4e70-ae8f-ec0356489278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02281fed3209406795711020e53b7162, user_id=3d9cade97a184adcad958e659e0ffcfc, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:46.806173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943da2db-2e0b-4e70-ae8f-ec0356489278 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:46.836448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1026/2028] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:02:46 2022] PUT /identity/v3/projects/02281fed3209406795711020e53b7162/users/3d9cade97a184adcad958e659e0ffcfc/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:02:46.837841 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ef753-c030-4ecc-b41c-e0478d15a6f6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:46.838328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ef753-c030-4ecc-b41c-e0478d15a6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:46.838411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ef753-c030-4ecc-b41c-e0478d15a6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:46.912156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-dc6ef753-c030-4ecc-b41c-e0478d15a6f6 None None] MFA Rules not processed for user `3d9cade97a184adcad958e659e0ffcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:46.913764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dc6ef753-c030-4ecc-b41c-e0478d15a6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:46.958802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943da2db-2e0b-4e70-ae8f-ec0356489278 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:46.960714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-943da2db-2e0b-4e70-ae8f-ec0356489278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:47.037932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943da2db-2e0b-4e70-ae8f-ec0356489278 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:47.042173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d2d2a-2bd1-44b6-a314-23917b337ed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:47.042969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d2d2a-2bd1-44b6-a314-23917b337ed3 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:47.043301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d2d2a-2bd1-44b6-a314-23917b337ed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:47.043576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d2d2a-2bd1-44b6-a314-23917b337ed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:47.045503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-435d2d2a-2bd1-44b6-a314-23917b337ed3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:47.053370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1003/2029] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:46 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:47.065524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ef753-c030-4ecc-b41c-e0478d15a6f6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:47.067957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dc6ef753-c030-4ecc-b41c-e0478d15a6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:47.083248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ef753-c030-4ecc-b41c-e0478d15a6f6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:47.087359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d9459-7d6b-44c0-8cf3-eb2e4cae004b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:47.088104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d9459-7d6b-44c0-8cf3-eb2e4cae004b service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:47.088389 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d9459-7d6b-44c0-8cf3-eb2e4cae004b service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:47.088665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d9459-7d6b-44c0-8cf3-eb2e4cae004b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:47.098445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d9459-7d6b-44c0-8cf3-eb2e4cae004b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:47.099946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d9459-7d6b-44c0-8cf3-eb2e4cae004b service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:47.112658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1004/2030] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:47 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:02:47.155978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d2d2a-2bd1-44b6-a314-23917b337ed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:47.157794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d2d2a-2bd1-44b6-a314-23917b337ed3 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:47.179578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1027/2031] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:47.274755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b8f5-c239-4f58-9c68-bc3f109c69e4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:47.275053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b8f5-c239-4f58-9c68-bc3f109c69e4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:47.275385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b8f5-c239-4f58-9c68-bc3f109c69e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:47.332096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-82d3b8f5-c239-4f58-9c68-bc3f109c69e4 None None] MFA Rules not processed for user `41227378a57c43219e4ae460c642f834`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:47.333248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-82d3b8f5-c239-4f58-9c68-bc3f109c69e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:47.368674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d2d2a-2bd1-44b6-a314-23917b337ed3 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:47.384313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d2d2a-2bd1-44b6-a314-23917b337ed3 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:47.389777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f8a3c-1317-49db-95bd-1f4e07cc8c23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:47.390701 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8a3c-1317-49db-95bd-1f4e07cc8c23 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:47.391109 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8a3c-1317-49db-95bd-1f4e07cc8c23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:47.391699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8a3c-1317-49db-95bd-1f4e07cc8c23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:47.464005 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1005/2032] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:47.504197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f8a3c-1317-49db-95bd-1f4e07cc8c23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:47.505664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f8a3c-1317-49db-95bd-1f4e07cc8c23 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:47.537597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1028/2033] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:47 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:02:47.572902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3b8f5-c239-4f58-9c68-bc3f109c69e4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:47.579816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:47.588595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3b8f5-c239-4f58-9c68-bc3f109c69e4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:47.595587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e318a-5495-4f2f-bb01-e09e0e3cda54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:47.596335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e318a-5495-4f2f-bb01-e09e0e3cda54 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:47.596690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e318a-5495-4f2f-bb01-e09e0e3cda54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:47.597068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e318a-5495-4f2f-bb01-e09e0e3cda54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:47.609953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e318a-5495-4f2f-bb01-e09e0e3cda54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:47.611703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e318a-5495-4f2f-bb01-e09e0e3cda54 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:47.655261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1006/2034] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:53.509828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e8c83-8bd9-44f4-befb-70efda49d026 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:53.510358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e8c83-8bd9-44f4-befb-70efda49d026 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:53.510468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e8c83-8bd9-44f4-befb-70efda49d026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:53.530077 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ab9e8c83-8bd9-44f4-befb-70efda49d026 None None] MFA Rules not processed for user `41227378a57c43219e4ae460c642f834`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:53.531347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ab9e8c83-8bd9-44f4-befb-70efda49d026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:53.538120 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1029/2035] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:02:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:57.628143 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:03.524529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:09.154487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e318a-5495-4f2f-bb01-e09e0e3cda54 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:09.257007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e318a-5495-4f2f-bb01-e09e0e3cda54 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:09.261718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:09.262362 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:09.262640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:09.262916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] PATH_INFO: `/v3/users/3a6ba03e851d4d778b2a63bfbf440616` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:09.265108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a6ba03e851d4d778b2a63bfbf440616)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:09.266884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:09.323883 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a6ba03e851d4d778b2a63bfbf440616'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:09.337494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a6ba03e851d4d778b2a63bfbf440616'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:09.338250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a6ba03e851d4d778b2a63bfbf440616'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:09.338807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a6ba03e851d4d778b2a63bfbf440616'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:09.347717 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a6ba03e851d4d778b2a63bfbf440616'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:09.363182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1007/2036] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:09 2022] DELETE /identity/v3/users/3a6ba03e851d4d778b2a63bfbf440616 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:09.371658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82a286-1661-46f5-8f06-1bc4accaa424 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:09.372112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82a286-1661-46f5-8f06-1bc4accaa424 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:09.372283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82a286-1661-46f5-8f06-1bc4accaa424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:09.406744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-de82a286-1661-46f5-8f06-1bc4accaa424 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:09.407998 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-de82a286-1661-46f5-8f06-1bc4accaa424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:09.501267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1030/2037] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:09.510114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:09.511578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:09.614382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d451947-52ae-4f22-bac1-ebd2e6b971f1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:09.618564 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5441c-647c-40fb-b462-66f065122c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:09.619313 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5441c-647c-40fb-b462-66f065122c3f service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:09.619664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5441c-647c-40fb-b462-66f065122c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:09.620104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5441c-647c-40fb-b462-66f065122c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:09.630087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5441c-647c-40fb-b462-66f065122c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:09.631598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5441c-647c-40fb-b462-66f065122c3f service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:09.642586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1008/2038] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:10.141350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82a286-1661-46f5-8f06-1bc4accaa424 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:10.147668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-de82a286-1661-46f5-8f06-1bc4accaa424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:10.161101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82a286-1661-46f5-8f06-1bc4accaa424 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:10.167107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:10.167707 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:10.167988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:10.168263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] PATH_INFO: `/v3/projects/06d10e93c76a44ed8e741aa6cdf6457f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:10.169918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06d10e93c76a44ed8e741aa6cdf6457f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:10.171921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:10.234527 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d10e93c76a44ed8e741aa6cdf6457f', 'request_id': 'req-b6fc1dfd-8c59-409c-871a-b24b197b53c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.245207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d10e93c76a44ed8e741aa6cdf6457f', 'request_id': 'req-b6fc1dfd-8c59-409c-871a-b24b197b53c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.253545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d10e93c76a44ed8e741aa6cdf6457f', 'request_id': 'req-b6fc1dfd-8c59-409c-871a-b24b197b53c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.265495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] The token cache is being invalidate because project 06d10e93c76a44ed8e741aa6cdf6457f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:10.265791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.266457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.271815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1031/2039] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:10 2022] DELETE /identity/v3/projects/06d10e93c76a44ed8e741aa6cdf6457f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:10.273395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5441c-647c-40fb-b462-66f065122c3f service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:10.283871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5441c-647c-40fb-b462-66f065122c3f service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:10.302840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:10.351351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b72b92-c6ff-463b-a2f2-cdc21863dca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:10.351907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b72b92-c6ff-463b-a2f2-cdc21863dca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:10.352190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b72b92-c6ff-463b-a2f2-cdc21863dca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:10.352464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b72b92-c6ff-463b-a2f2-cdc21863dca8 admin admin] PATH_INFO: `/v3/users/3d9cade97a184adcad958e659e0ffcfc` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:10.353618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b72b92-c6ff-463b-a2f2-cdc21863dca8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d9cade97a184adcad958e659e0ffcfc)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:10.355668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b72b92-c6ff-463b-a2f2-cdc21863dca8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:10.417868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-23b72b92-c6ff-463b-a2f2-cdc21863dca8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d9cade97a184adcad958e659e0ffcfc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.430421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-23b72b92-c6ff-463b-a2f2-cdc21863dca8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d9cade97a184adcad958e659e0ffcfc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.431145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-23b72b92-c6ff-463b-a2f2-cdc21863dca8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d9cade97a184adcad958e659e0ffcfc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.431761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-23b72b92-c6ff-463b-a2f2-cdc21863dca8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d9cade97a184adcad958e659e0ffcfc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.440573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-23b72b92-c6ff-463b-a2f2-cdc21863dca8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d9cade97a184adcad958e659e0ffcfc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.454172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1009/2040] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:10 2022] DELETE /identity/v3/users/3d9cade97a184adcad958e659e0ffcfc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:10.802451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:10.804295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:10.818341 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc1dfd-8c59-409c-871a-b24b197b53c4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:10.887646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:10.888286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:10.888566 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:10.888903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] PATH_INFO: `/v3/projects/02281fed3209406795711020e53b7162` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:10.890018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02281fed3209406795711020e53b7162)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:10.891979 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:10.923010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:10.940528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02281fed3209406795711020e53b7162', 'request_id': 'req-5d78ffb2-8edf-434d-976e-1b5e78553b81'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.949052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02281fed3209406795711020e53b7162', 'request_id': 'req-5d78ffb2-8edf-434d-976e-1b5e78553b81'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.955572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02281fed3209406795711020e53b7162', 'request_id': 'req-5d78ffb2-8edf-434d-976e-1b5e78553b81'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.967527 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] The token cache is being invalidate because project 02281fed3209406795711020e53b7162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:10.967933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.968556 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d78ffb2-8edf-434d-976e-1b5e78553b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:10.973283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1032/2041] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:10 2022] DELETE /identity/v3/projects/02281fed3209406795711020e53b7162 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:10.975043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a97896-6135-4fa8-acc7-830818f415b7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:10.975338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a97896-6135-4fa8-acc7-830818f415b7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:10.975615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a97896-6135-4fa8-acc7-830818f415b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:10.995888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-04a97896-6135-4fa8-acc7-830818f415b7 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:10.997252 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-04a97896-6135-4fa8-acc7-830818f415b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:11.059989 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1010/2042] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:11.068300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363cbac-a56e-44e1-ac49-59da2dbe27d2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:11.068587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363cbac-a56e-44e1-ac49-59da2dbe27d2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:11.068858 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363cbac-a56e-44e1-ac49-59da2dbe27d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:11.090060 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4363cbac-a56e-44e1-ac49-59da2dbe27d2 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:11.091302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4363cbac-a56e-44e1-ac49-59da2dbe27d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:11.105850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1033/2043] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:11.107268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a97896-6135-4fa8-acc7-830818f415b7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:11.122817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a97896-6135-4fa8-acc7-830818f415b7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:11.143860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1967e-d113-435e-bd3f-ed44eee6a8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:11.146409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1967e-d113-435e-bd3f-ed44eee6a8de admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:11.147283 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1967e-d113-435e-bd3f-ed44eee6a8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:11.147619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1967e-d113-435e-bd3f-ed44eee6a8de admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:11.149431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1967e-d113-435e-bd3f-ed44eee6a8de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:03:11.149763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1967e-d113-435e-bd3f-ed44eee6a8de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:11.152209 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1967e-d113-435e-bd3f-ed44eee6a8de admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:11.175994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1011/2044] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:03:11.182565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363cbac-a56e-44e1-ac49-59da2dbe27d2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:11.192326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363cbac-a56e-44e1-ac49-59da2dbe27d2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:11.197368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd10e2-e4b4-493c-b194-ea5da36d363b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:11.197978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd10e2-e4b4-493c-b194-ea5da36d363b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:11.198269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd10e2-e4b4-493c-b194-ea5da36d363b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:11.198650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd10e2-e4b4-493c-b194-ea5da36d363b admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:11.199382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd10e2-e4b4-493c-b194-ea5da36d363b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:11.201816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd10e2-e4b4-493c-b194-ea5da36d363b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:11.240201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1034/2045] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:11 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:11.244338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1967e-d113-435e-bd3f-ed44eee6a8de admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:11.255415 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1967e-d113-435e-bd3f-ed44eee6a8de admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:11.282102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:11.313989 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a6bee-c7e5-400b-b262-276811619249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:11.314595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a6bee-c7e5-400b-b262-276811619249 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:11.314873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a6bee-c7e5-400b-b262-276811619249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:11.315179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a6bee-c7e5-400b-b262-276811619249 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:11.315804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a6bee-c7e5-400b-b262-276811619249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:11.317708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a6bee-c7e5-400b-b262-276811619249 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:11.338308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1012/2046] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:11 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:11.342457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd10e2-e4b4-493c-b194-ea5da36d363b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:11.352647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd10e2-e4b4-493c-b194-ea5da36d363b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:11.357258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186fcfa-a62b-4c47-9eab-ce4bb5c2f220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:11.358382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186fcfa-a62b-4c47-9eab-ce4bb5c2f220 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:11.358663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186fcfa-a62b-4c47-9eab-ce4bb5c2f220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:11.358933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186fcfa-a62b-4c47-9eab-ce4bb5c2f220 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:11.359597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186fcfa-a62b-4c47-9eab-ce4bb5c2f220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:11.361175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186fcfa-a62b-4c47-9eab-ce4bb5c2f220 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:11.370927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1035/2047] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:03:11.375455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a6bee-c7e5-400b-b262-276811619249 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:11.385692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a6bee-c7e5-400b-b262-276811619249 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:11.390614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc90ed-8954-49d8-b606-1748e9e51c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:11.391250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc90ed-8954-49d8-b606-1748e9e51c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:11.391538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc90ed-8954-49d8-b606-1748e9e51c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:11.391812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc90ed-8954-49d8-b606-1748e9e51c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:11.392488 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc90ed-8954-49d8-b606-1748e9e51c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:11.394215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc90ed-8954-49d8-b606-1748e9e51c2b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:11.406595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1013/2048] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:11.410929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186fcfa-a62b-4c47-9eab-ce4bb5c2f220 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:11.421417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186fcfa-a62b-4c47-9eab-ce4bb5c2f220 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:11.425948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882f7b0-0688-4ccf-ac8b-b1aa793f813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:11.426592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882f7b0-0688-4ccf-ac8b-b1aa793f813b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:11.426870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882f7b0-0688-4ccf-ac8b-b1aa793f813b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:11.427171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882f7b0-0688-4ccf-ac8b-b1aa793f813b admin admin] PATH_INFO: `/v3/projects/46cdba3dc336405ab5d5866a0a5d5c99/users/d6fe8cd5456b48789025e206c3ddb1e7/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:11.442369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882f7b0-0688-4ccf-ac8b-b1aa793f813b admin admin] RBAC: Authorizing `identity:create_grant(project_id=46cdba3dc336405ab5d5866a0a5d5c99, user_id=d6fe8cd5456b48789025e206c3ddb1e7, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:11.445285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882f7b0-0688-4ccf-ac8b-b1aa793f813b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:11.459852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1036/2049] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:11 2022] PUT /identity/v3/projects/46cdba3dc336405ab5d5866a0a5d5c99/users/d6fe8cd5456b48789025e206c3ddb1e7/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:11.471542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fa95e-6ae8-44c3-854f-740b7492b759 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:11.471837 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fa95e-6ae8-44c3-854f-740b7492b759 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:11.472118 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fa95e-6ae8-44c3-854f-740b7492b759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:11.500591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-276fa95e-6ae8-44c3-854f-740b7492b759 None None] MFA Rules not processed for user `d6fe8cd5456b48789025e206c3ddb1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:11.501599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-276fa95e-6ae8-44c3-854f-740b7492b759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:11.571812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1014/2050] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:11.583631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882f7b0-0688-4ccf-ac8b-b1aa793f813b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:11.585329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6882f7b0-0688-4ccf-ac8b-b1aa793f813b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:11.643187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882f7b0-0688-4ccf-ac8b-b1aa793f813b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:11.646900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024b07e-bc36-4229-ab6f-1c75bd7e8f6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:11.647535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024b07e-bc36-4229-ab6f-1c75bd7e8f6d service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:11.647816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024b07e-bc36-4229-ab6f-1c75bd7e8f6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:11.648088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024b07e-bc36-4229-ab6f-1c75bd7e8f6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:11.656742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024b07e-bc36-4229-ab6f-1c75bd7e8f6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:11.658169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024b07e-bc36-4229-ab6f-1c75bd7e8f6d service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:11.667100 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1037/2051] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:11.689683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fa95e-6ae8-44c3-854f-740b7492b759 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:11.751308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fa95e-6ae8-44c3-854f-740b7492b759 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:11.755807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:11.756428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:11.756705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:11.756975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] PATH_INFO: `/v3/users/d6fe8cd5456b48789025e206c3ddb1e7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:11.758074 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6fe8cd5456b48789025e206c3ddb1e7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:11.760049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:11.799067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:11.805390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6fe8cd5456b48789025e206c3ddb1e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:11.820834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6fe8cd5456b48789025e206c3ddb1e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:11.821567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6fe8cd5456b48789025e206c3ddb1e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:11.822264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6fe8cd5456b48789025e206c3ddb1e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:11.831267 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6fe8cd5456b48789025e206c3ddb1e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:11.843854 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1015/2052] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:11 2022] DELETE /identity/v3/users/d6fe8cd5456b48789025e206c3ddb1e7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:11.848412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b102d-1430-4fe9-8488-80866ad01db5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:11.848901 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b102d-1430-4fe9-8488-80866ad01db5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:11.849014 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b102d-1430-4fe9-8488-80866ad01db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:11.869678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0a3b102d-1430-4fe9-8488-80866ad01db5 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:11.870896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0a3b102d-1430-4fe9-8488-80866ad01db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:11.919772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:11.936501 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1038/2053] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:11.947015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:11.949384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:12.006482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7c172-e9af-415f-9072-cce493e7ed79 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:12.010625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377fd07-8da1-4106-81e0-d4c0e0504c61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:12.011258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377fd07-8da1-4106-81e0-d4c0e0504c61 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:12.011541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377fd07-8da1-4106-81e0-d4c0e0504c61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:12.011810 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377fd07-8da1-4106-81e0-d4c0e0504c61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:12.020571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377fd07-8da1-4106-81e0-d4c0e0504c61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:12.021972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377fd07-8da1-4106-81e0-d4c0e0504c61 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:12.031321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1016/2054] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:12.309583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b102d-1430-4fe9-8488-80866ad01db5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:12.311596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0a3b102d-1430-4fe9-8488-80866ad01db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:12.350186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b102d-1430-4fe9-8488-80866ad01db5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:12.357668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:12.358322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:12.358687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:12.358879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] PATH_INFO: `/v3/projects/46cdba3dc336405ab5d5866a0a5d5c99` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:12.360105 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=46cdba3dc336405ab5d5866a0a5d5c99)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:12.362062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:12.421420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46cdba3dc336405ab5d5866a0a5d5c99', 'request_id': 'req-ce7e9f98-e6f1-4623-875b-c2b894a122ff'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:12.431949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46cdba3dc336405ab5d5866a0a5d5c99', 'request_id': 'req-ce7e9f98-e6f1-4623-875b-c2b894a122ff'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:12.443425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46cdba3dc336405ab5d5866a0a5d5c99', 'request_id': 'req-ce7e9f98-e6f1-4623-875b-c2b894a122ff'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:12.457321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] The token cache is being invalidate because project 46cdba3dc336405ab5d5866a0a5d5c99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:12.457816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:12.458448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ce7e9f98-e6f1-4623-875b-c2b894a122ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:12.460524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1039/2055] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:12 2022] DELETE /identity/v3/projects/46cdba3dc336405ab5d5866a0a5d5c99 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:12.465805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6a813-dff7-4763-935d-e6f379bed595 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:12.466352 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6a813-dff7-4763-935d-e6f379bed595 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:12.466531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6a813-dff7-4763-935d-e6f379bed595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:12.492340 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e2f6a813-dff7-4763-935d-e6f379bed595 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:12.493662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e2f6a813-dff7-4763-935d-e6f379bed595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:12.569327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1017/2056] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:12.577827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e9a91-8a33-48bc-b2e4-0488b2a32a59 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:12.578112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e9a91-8a33-48bc-b2e4-0488b2a32a59 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:12.578386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e9a91-8a33-48bc-b2e4-0488b2a32a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:12.598700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-087e9a91-8a33-48bc-b2e4-0488b2a32a59 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:12.600026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-087e9a91-8a33-48bc-b2e4-0488b2a32a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:12.612634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1040/2057] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:12.613900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6a813-dff7-4763-935d-e6f379bed595 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:12.624232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6a813-dff7-4763-935d-e6f379bed595 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:12.628827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d603168-e1be-4a55-a836-8ed3b1f6985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:12.629440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d603168-e1be-4a55-a836-8ed3b1f6985c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:12.629720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d603168-e1be-4a55-a836-8ed3b1f6985c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:12.629998 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d603168-e1be-4a55-a836-8ed3b1f6985c admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:12.630661 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d603168-e1be-4a55-a836-8ed3b1f6985c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:03:12.630948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d603168-e1be-4a55-a836-8ed3b1f6985c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:12.632557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d603168-e1be-4a55-a836-8ed3b1f6985c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:12.644631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1018/2058] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:03:12.649064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e9a91-8a33-48bc-b2e4-0488b2a32a59 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:12.658672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e9a91-8a33-48bc-b2e4-0488b2a32a59 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:12.663384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c804d24-f840-4e0f-8e21-54e643689e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:12.663980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c804d24-f840-4e0f-8e21-54e643689e41 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:12.664261 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c804d24-f840-4e0f-8e21-54e643689e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:12.664534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c804d24-f840-4e0f-8e21-54e643689e41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:12.665170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c804d24-f840-4e0f-8e21-54e643689e41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:12.667245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c804d24-f840-4e0f-8e21-54e643689e41 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:12.722226 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1041/2059] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:12 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:12.728617 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d603168-e1be-4a55-a836-8ed3b1f6985c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:12.747530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d603168-e1be-4a55-a836-8ed3b1f6985c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:12.798589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:12.814155 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c652e-d7ee-44a2-8a28-d7c15b8f14e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:12.814768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c652e-d7ee-44a2-8a28-d7c15b8f14e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:12.815088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c652e-d7ee-44a2-8a28-d7c15b8f14e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:12.815368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c652e-d7ee-44a2-8a28-d7c15b8f14e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:12.815995 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c652e-d7ee-44a2-8a28-d7c15b8f14e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:12.818206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c652e-d7ee-44a2-8a28-d7c15b8f14e3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:12.838745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1019/2060] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:12 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:03:12.843136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c804d24-f840-4e0f-8e21-54e643689e41 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:12.853916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c804d24-f840-4e0f-8e21-54e643689e41 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:12.859793 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409af314-19d9-4fa9-a63d-e1bd1cf99277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:12.860443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409af314-19d9-4fa9-a63d-e1bd1cf99277 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:12.860721 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409af314-19d9-4fa9-a63d-e1bd1cf99277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:12.860992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409af314-19d9-4fa9-a63d-e1bd1cf99277 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:12.861628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409af314-19d9-4fa9-a63d-e1bd1cf99277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:12.863282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409af314-19d9-4fa9-a63d-e1bd1cf99277 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:12.874054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1042/2061] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:12.878226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c652e-d7ee-44a2-8a28-d7c15b8f14e3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:12.888400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c652e-d7ee-44a2-8a28-d7c15b8f14e3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:12.893312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb30fb-cc28-4250-8ebc-23efa348e08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:12.893901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb30fb-cc28-4250-8ebc-23efa348e08c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:12.894177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb30fb-cc28-4250-8ebc-23efa348e08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:12.894445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb30fb-cc28-4250-8ebc-23efa348e08c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:12.895111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb30fb-cc28-4250-8ebc-23efa348e08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:12.896722 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb30fb-cc28-4250-8ebc-23efa348e08c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:12.907517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1020/2062] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:12.911670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409af314-19d9-4fa9-a63d-e1bd1cf99277 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:12.922793 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409af314-19d9-4fa9-a63d-e1bd1cf99277 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:12.927850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7085a-dcf1-4807-abab-c80eaff4935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:12.928451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7085a-dcf1-4807-abab-c80eaff4935b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:12.928732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7085a-dcf1-4807-abab-c80eaff4935b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:12.929007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7085a-dcf1-4807-abab-c80eaff4935b admin admin] PATH_INFO: `/v3/projects/7017520884ec48c38ea1cb3011d10300/users/c5a33f871d944505a07b8ef55d4e0f3e/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:12.944955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7085a-dcf1-4807-abab-c80eaff4935b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7017520884ec48c38ea1cb3011d10300, user_id=c5a33f871d944505a07b8ef55d4e0f3e, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:12.947873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7085a-dcf1-4807-abab-c80eaff4935b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:12.960227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1043/2063] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:12 2022] PUT /identity/v3/projects/7017520884ec48c38ea1cb3011d10300/users/c5a33f871d944505a07b8ef55d4e0f3e/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:12.972222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a52190-875b-49ac-9eff-a9ad964527d7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:12.972520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a52190-875b-49ac-9eff-a9ad964527d7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:12.972816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a52190-875b-49ac-9eff-a9ad964527d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:13.005178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-79a52190-875b-49ac-9eff-a9ad964527d7 None None] MFA Rules not processed for user `c5a33f871d944505a07b8ef55d4e0f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:13.006494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-79a52190-875b-49ac-9eff-a9ad964527d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:13.080833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1021/2064] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:13.085367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7085a-dcf1-4807-abab-c80eaff4935b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:13.159333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7085a-dcf1-4807-abab-c80eaff4935b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:13.164144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169ac70-6510-4b63-9001-f0592accfc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:13.164748 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169ac70-6510-4b63-9001-f0592accfc53 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:13.165034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169ac70-6510-4b63-9001-f0592accfc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:13.165302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169ac70-6510-4b63-9001-f0592accfc53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:13.165943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169ac70-6510-4b63-9001-f0592accfc53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:13.167868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169ac70-6510-4b63-9001-f0592accfc53 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:13.188477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:13.197168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1044/2065] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:13 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:13.201300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a52190-875b-49ac-9eff-a9ad964527d7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:13.212082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a52190-875b-49ac-9eff-a9ad964527d7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:13.267153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6beda-6fc1-4266-a3d9-4fae83b239b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:13.267757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6beda-6fc1-4266-a3d9-4fae83b239b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:13.268033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6beda-6fc1-4266-a3d9-4fae83b239b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:13.268303 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6beda-6fc1-4266-a3d9-4fae83b239b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:13.268913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6beda-6fc1-4266-a3d9-4fae83b239b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:13.270842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6beda-6fc1-4266-a3d9-4fae83b239b2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:13.293552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1022/2066] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:13 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:13.298383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169ac70-6510-4b63-9001-f0592accfc53 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:13.309608 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169ac70-6510-4b63-9001-f0592accfc53 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:13.314622 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928327b5-4b2e-4fdc-8a37-c65707784657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:13.315254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928327b5-4b2e-4fdc-8a37-c65707784657 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:13.315531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928327b5-4b2e-4fdc-8a37-c65707784657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:13.315797 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928327b5-4b2e-4fdc-8a37-c65707784657 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:13.316430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928327b5-4b2e-4fdc-8a37-c65707784657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:13.318330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928327b5-4b2e-4fdc-8a37-c65707784657 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:13.328942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1045/2067] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:13.333141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6beda-6fc1-4266-a3d9-4fae83b239b2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:13.343661 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6beda-6fc1-4266-a3d9-4fae83b239b2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:13.348504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9486d2f-0cad-4845-85c8-5a30f8cefd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:13.349158 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486d2f-0cad-4845-85c8-5a30f8cefd6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:13.349441 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486d2f-0cad-4845-85c8-5a30f8cefd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:13.349719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486d2f-0cad-4845-85c8-5a30f8cefd6f admin admin] PATH_INFO: `/v3/projects/e878d7a5045b493dae1d70565f14708c/users/7cf4d856d0504cffa2829ae143706d4b/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:13.364970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9486d2f-0cad-4845-85c8-5a30f8cefd6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e878d7a5045b493dae1d70565f14708c, user_id=7cf4d856d0504cffa2829ae143706d4b, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:13.367735 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9486d2f-0cad-4845-85c8-5a30f8cefd6f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:13.380551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1023/2068] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:13 2022] PUT /identity/v3/projects/e878d7a5045b493dae1d70565f14708c/users/7cf4d856d0504cffa2829ae143706d4b/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:13.384443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928327b5-4b2e-4fdc-8a37-c65707784657 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:13.395392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928327b5-4b2e-4fdc-8a37-c65707784657 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:13.453398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df42ab-d288-43bd-8a2a-2b2312804d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:13.453929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df42ab-d288-43bd-8a2a-2b2312804d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:13.454214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df42ab-d288-43bd-8a2a-2b2312804d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:13.454495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df42ab-d288-43bd-8a2a-2b2312804d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:13.455182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df42ab-d288-43bd-8a2a-2b2312804d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:13.456788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df42ab-d288-43bd-8a2a-2b2312804d21 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:13.467818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1046/2069] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:13.472090 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9486d2f-0cad-4845-85c8-5a30f8cefd6f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:13.483176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9486d2f-0cad-4845-85c8-5a30f8cefd6f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:13.487756 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c147e0d-d9d5-42e8-86b4-327db3c889cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:13.488412 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c147e0d-d9d5-42e8-86b4-327db3c889cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:13.488692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c147e0d-d9d5-42e8-86b4-327db3c889cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:13.488966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c147e0d-d9d5-42e8-86b4-327db3c889cf admin admin] PATH_INFO: `/v3/projects/e878d7a5045b493dae1d70565f14708c/users/7cf4d856d0504cffa2829ae143706d4b/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:13.490865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c147e0d-d9d5-42e8-86b4-327db3c889cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e878d7a5045b493dae1d70565f14708c, user_id=7cf4d856d0504cffa2829ae143706d4b, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:13.493541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c147e0d-d9d5-42e8-86b4-327db3c889cf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:13.505565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1024/2070] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:13 2022] PUT /identity/v3/projects/e878d7a5045b493dae1d70565f14708c/users/7cf4d856d0504cffa2829ae143706d4b/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:13.515782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a405f-1241-40a8-9441-551969b14052 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:13.516099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a405f-1241-40a8-9441-551969b14052 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:13.516402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a405f-1241-40a8-9441-551969b14052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:13.552156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-db6a405f-1241-40a8-9441-551969b14052 None None] MFA Rules not processed for user `7cf4d856d0504cffa2829ae143706d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:13.553216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db6a405f-1241-40a8-9441-551969b14052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:13.623893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1047/2071] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:13.635563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c147e0d-d9d5-42e8-86b4-327db3c889cf admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:13.637442 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1c147e0d-d9d5-42e8-86b4-327db3c889cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:13.647202 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:13.702114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c147e0d-d9d5-42e8-86b4-327db3c889cf admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:13.706057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d09fa5-6f64-4a9f-b51b-82c0c8665790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:13.706671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d09fa5-6f64-4a9f-b51b-82c0c8665790 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:13.706953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d09fa5-6f64-4a9f-b51b-82c0c8665790 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:13.707266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d09fa5-6f64-4a9f-b51b-82c0c8665790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:13.768649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d09fa5-6f64-4a9f-b51b-82c0c8665790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:13.769813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d09fa5-6f64-4a9f-b51b-82c0c8665790 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:13.790465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1025/2072] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:22.669110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a405f-1241-40a8-9441-551969b14052 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:22.738365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a405f-1241-40a8-9441-551969b14052 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:22.742925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77d0b3-1af0-4a7b-b039-f45eb588d3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:22.743597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d0b3-1af0-4a7b-b039-f45eb588d3be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:22.743880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d0b3-1af0-4a7b-b039-f45eb588d3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:22.744157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d0b3-1af0-4a7b-b039-f45eb588d3be admin admin] PATH_INFO: `/v3/users/c5a33f871d944505a07b8ef55d4e0f3e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:22.745282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77d0b3-1af0-4a7b-b039-f45eb588d3be admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a33f871d944505a07b8ef55d4e0f3e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:22.747236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77d0b3-1af0-4a7b-b039-f45eb588d3be admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:22.761981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:22.812852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d09fa5-6f64-4a9f-b51b-82c0c8665790 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:22.814411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d2d09fa5-6f64-4a9f-b51b-82c0c8665790 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:22.816849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d77d0b3-1af0-4a7b-b039-f45eb588d3be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a33f871d944505a07b8ef55d4e0f3e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:22.817604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d77d0b3-1af0-4a7b-b039-f45eb588d3be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a33f871d944505a07b8ef55d4e0f3e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:22.829540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d09fa5-6f64-4a9f-b51b-82c0c8665790 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:22.831020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d77d0b3-1af0-4a7b-b039-f45eb588d3be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a33f871d944505a07b8ef55d4e0f3e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:22.834350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629be8c0-c018-4bf7-bc2c-474aa9fef929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:22.834983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be8c0-c018-4bf7-bc2c-474aa9fef929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:22.835295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be8c0-c018-4bf7-bc2c-474aa9fef929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:22.835600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be8c0-c018-4bf7-bc2c-474aa9fef929 admin admin] PATH_INFO: `/v3/users/fa8c72b69f3f45e49eef5fa9b8f60f05` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:22.836789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629be8c0-c018-4bf7-bc2c-474aa9fef929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa8c72b69f3f45e49eef5fa9b8f60f05)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:22.838811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d77d0b3-1af0-4a7b-b039-f45eb588d3be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a33f871d944505a07b8ef55d4e0f3e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:22.838935 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629be8c0-c018-4bf7-bc2c-474aa9fef929 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:22.846339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d77d0b3-1af0-4a7b-b039-f45eb588d3be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a33f871d944505a07b8ef55d4e0f3e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:22.852401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1048/2073] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:22 2022] DELETE /identity/v3/users/c5a33f871d944505a07b8ef55d4e0f3e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:22.857553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b722d5-f4fb-4dcc-88b1-5a183efa8c5b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:22.857843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b722d5-f4fb-4dcc-88b1-5a183efa8c5b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:22.858122 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b722d5-f4fb-4dcc-88b1-5a183efa8c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:22.878043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-25b722d5-f4fb-4dcc-88b1-5a183efa8c5b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:22.879286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-25b722d5-f4fb-4dcc-88b1-5a183efa8c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:22.889965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-629be8c0-c018-4bf7-bc2c-474aa9fef929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa8c72b69f3f45e49eef5fa9b8f60f05'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:22.902800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-629be8c0-c018-4bf7-bc2c-474aa9fef929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa8c72b69f3f45e49eef5fa9b8f60f05'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:22.903621 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-629be8c0-c018-4bf7-bc2c-474aa9fef929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa8c72b69f3f45e49eef5fa9b8f60f05'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:22.904261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-629be8c0-c018-4bf7-bc2c-474aa9fef929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa8c72b69f3f45e49eef5fa9b8f60f05'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:22.913075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-629be8c0-c018-4bf7-bc2c-474aa9fef929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa8c72b69f3f45e49eef5fa9b8f60f05'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:22.927235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1026/2074] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:22 2022] DELETE /identity/v3/users/fa8c72b69f3f45e49eef5fa9b8f60f05 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:22.931519 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e011ab-faba-452e-87a5-a5e2d5c6e190 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:22.931931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e011ab-faba-452e-87a5-a5e2d5c6e190 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:22.932244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e011ab-faba-452e-87a5-a5e2d5c6e190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:22.945708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1049/2075] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:03:22.953267 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-03e011ab-faba-452e-87a5-a5e2d5c6e190 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:22.953558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b722d5-f4fb-4dcc-88b1-5a183efa8c5b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:22.954498 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-03e011ab-faba-452e-87a5-a5e2d5c6e190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:22.954897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-25b722d5-f4fb-4dcc-88b1-5a183efa8c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:22.985766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:23.015079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b722d5-f4fb-4dcc-88b1-5a183efa8c5b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:23.026125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1027/2076] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:23.032766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e011ab-faba-452e-87a5-a5e2d5c6e190 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:23.056366 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:23.066457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c66b86-27bf-4c52-bfa5-fc54bb6a05b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:23.067294 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c66b86-27bf-4c52-bfa5-fc54bb6a05b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:23.067745 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c66b86-27bf-4c52-bfa5-fc54bb6a05b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:23.068394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c66b86-27bf-4c52-bfa5-fc54bb6a05b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:23.079189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c66b86-27bf-4c52-bfa5-fc54bb6a05b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:23.080626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c66b86-27bf-4c52-bfa5-fc54bb6a05b1 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:23.091549 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1050/2077] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:03:23.093533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e011ab-faba-452e-87a5-a5e2d5c6e190 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:23.097433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd03084-06d7-4ec2-b2b8-f97432fc6040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:23.098072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd03084-06d7-4ec2-b2b8-f97432fc6040 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:23.098357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd03084-06d7-4ec2-b2b8-f97432fc6040 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:23.098671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd03084-06d7-4ec2-b2b8-f97432fc6040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:23.109468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd03084-06d7-4ec2-b2b8-f97432fc6040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:23.110251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd03084-06d7-4ec2-b2b8-f97432fc6040 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:23.119998 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1028/2078] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:03:23.578516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c66b86-27bf-4c52-bfa5-fc54bb6a05b1 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:23.580187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f7c66b86-27bf-4c52-bfa5-fc54bb6a05b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:23.596715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c66b86-27bf-4c52-bfa5-fc54bb6a05b1 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:23.603176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:23.603957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:23.604320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:23.604608 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] PATH_INFO: `/v3/projects/7017520884ec48c38ea1cb3011d10300` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:23.606839 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7017520884ec48c38ea1cb3011d10300)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:23.610970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:23.632111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd03084-06d7-4ec2-b2b8-f97432fc6040 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:23.633576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0dd03084-06d7-4ec2-b2b8-f97432fc6040 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:23.654120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd03084-06d7-4ec2-b2b8-f97432fc6040 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:23.660301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:23.660943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:23.661223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:23.661527 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] PATH_INFO: `/v3/projects/bf2a001e8e6a4f7d8c8d7a49ad6eae6d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:23.662845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf2a001e8e6a4f7d8c8d7a49ad6eae6d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:23.664795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:23.727853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf2a001e8e6a4f7d8c8d7a49ad6eae6d', 'request_id': 'req-6bfdad32-bf6e-41ae-99a9-102ae47b134a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.735324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf2a001e8e6a4f7d8c8d7a49ad6eae6d', 'request_id': 'req-6bfdad32-bf6e-41ae-99a9-102ae47b134a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.739745 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7017520884ec48c38ea1cb3011d10300', 'request_id': 'req-86234b75-d319-45b7-96d7-da27786ae2a1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.741592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf2a001e8e6a4f7d8c8d7a49ad6eae6d', 'request_id': 'req-6bfdad32-bf6e-41ae-99a9-102ae47b134a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.749004 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7017520884ec48c38ea1cb3011d10300', 'request_id': 'req-86234b75-d319-45b7-96d7-da27786ae2a1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.756842 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7017520884ec48c38ea1cb3011d10300', 'request_id': 'req-86234b75-d319-45b7-96d7-da27786ae2a1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.762207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] The token cache is being invalidate because project bf2a001e8e6a4f7d8c8d7a49ad6eae6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:23.762207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.762882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.765264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1029/2079] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:23 2022] DELETE /identity/v3/projects/bf2a001e8e6a4f7d8c8d7a49ad6eae6d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:23.768419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] The token cache is being invalidate because project 7017520884ec48c38ea1cb3011d10300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:23.768780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.769498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.769684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:23.771132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1051/2080] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:23 2022] DELETE /identity/v3/projects/7017520884ec48c38ea1cb3011d10300 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:23.771293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:23.774677 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:23.775960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:23.864125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdad32-bf6e-41ae-99a9-102ae47b134a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:23.869545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:23.870181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:23.870460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:23.870729 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] PATH_INFO: `/v3/users/fdb0e67776374cee97ffad0b0d27070b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:23.871905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdb0e67776374cee97ffad0b0d27070b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:23.874159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:23.876785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:23.881687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86234b75-d319-45b7-96d7-da27786ae2a1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:23.906382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:23.907630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:23.907991 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:23.908338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] PATH_INFO: `/v3/users/7cf4d856d0504cffa2829ae143706d4b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:23.909589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf4d856d0504cffa2829ae143706d4b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:23.911641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:23.962319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdb0e67776374cee97ffad0b0d27070b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.979470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdb0e67776374cee97ffad0b0d27070b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.980374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdb0e67776374cee97ffad0b0d27070b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.981002 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdb0e67776374cee97ffad0b0d27070b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.989972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf4d856d0504cffa2829ae143706d4b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.990810 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdb0e67776374cee97ffad0b0d27070b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:23.991420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf4d856d0504cffa2829ae143706d4b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.015325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1030/2081] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:23 2022] DELETE /identity/v3/users/fdb0e67776374cee97ffad0b0d27070b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:24.015461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf4d856d0504cffa2829ae143706d4b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.021519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf4d856d0504cffa2829ae143706d4b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.023350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:24.035626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf4d856d0504cffa2829ae143706d4b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.044541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1052/2082] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:23 2022] DELETE /identity/v3/users/7cf4d856d0504cffa2829ae143706d4b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:24.409348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:24.411455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:24.504815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694677cd-b265-41a9-9701-00940556e7d3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:24.562127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:24.564140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:24.609190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:24.609853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:24.610140 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:24.610471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] PATH_INFO: `/v3/projects/e878d7a5045b493dae1d70565f14708c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:24.612161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e878d7a5045b493dae1d70565f14708c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:24.614246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:24.651141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:24.661066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35702ac-3913-46a5-bb24-4b6480209072 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:24.674557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e878d7a5045b493dae1d70565f14708c', 'request_id': 'req-957c9058-e462-40ae-be3e-eff81cdd8fa4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.694080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e878d7a5045b493dae1d70565f14708c', 'request_id': 'req-957c9058-e462-40ae-be3e-eff81cdd8fa4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.707935 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e878d7a5045b493dae1d70565f14708c', 'request_id': 'req-957c9058-e462-40ae-be3e-eff81cdd8fa4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.722071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] The token cache is being invalidate because project e878d7a5045b493dae1d70565f14708c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:24.722449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.723186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-957c9058-e462-40ae-be3e-eff81cdd8fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.724744 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1031/2083] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:24 2022] DELETE /identity/v3/projects/e878d7a5045b493dae1d70565f14708c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:24.735174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:24.735174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:24.735174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:24.735174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] PATH_INFO: `/v3/projects/494f3a99672747f99f55ee7ec723d2e4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:24.735683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=494f3a99672747f99f55ee7ec723d2e4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:24.735683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:24.784126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '494f3a99672747f99f55ee7ec723d2e4', 'request_id': 'req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.793458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '494f3a99672747f99f55ee7ec723d2e4', 'request_id': 'req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.801692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '494f3a99672747f99f55ee7ec723d2e4', 'request_id': 'req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.814936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] The token cache is being invalidate because project 494f3a99672747f99f55ee7ec723d2e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:24.815253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.815972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6aa6efa0-083a-4a77-bc01-becf0a8fdf6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:24.818172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1053/2084] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:24 2022] DELETE /identity/v3/projects/494f3a99672747f99f55ee7ec723d2e4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:24.822902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62cd9-02c1-4cfa-afb8-9c8ea9dbadc2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:24.823230 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62cd9-02c1-4cfa-afb8-9c8ea9dbadc2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:24.823510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62cd9-02c1-4cfa-afb8-9c8ea9dbadc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:24.844455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-62b62cd9-02c1-4cfa-afb8-9c8ea9dbadc2 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:24.845734 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-62b62cd9-02c1-4cfa-afb8-9c8ea9dbadc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:24.912943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1032/2085] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:24.921699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892a6e9-5663-48fe-a8bb-44f31e2f714b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:24.922001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892a6e9-5663-48fe-a8bb-44f31e2f714b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:24.922279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892a6e9-5663-48fe-a8bb-44f31e2f714b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:24.944436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5892a6e9-5663-48fe-a8bb-44f31e2f714b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:24.945952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5892a6e9-5663-48fe-a8bb-44f31e2f714b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:24.954401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1054/2086] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:24.958522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b62cd9-02c1-4cfa-afb8-9c8ea9dbadc2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:24.970207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b62cd9-02c1-4cfa-afb8-9c8ea9dbadc2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:24.975197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28d4dd-59de-4cf8-9bde-f312e8ff3e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:24.975804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28d4dd-59de-4cf8-9bde-f312e8ff3e19 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:24.976083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28d4dd-59de-4cf8-9bde-f312e8ff3e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:24.976357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28d4dd-59de-4cf8-9bde-f312e8ff3e19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:24.977011 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28d4dd-59de-4cf8-9bde-f312e8ff3e19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:03:24.977285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28d4dd-59de-4cf8-9bde-f312e8ff3e19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:24.978864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28d4dd-59de-4cf8-9bde-f312e8ff3e19 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:24.995185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1033/2087] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:03:24.999636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892a6e9-5663-48fe-a8bb-44f31e2f714b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.009786 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892a6e9-5663-48fe-a8bb-44f31e2f714b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:25.017267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130eb80-5543-4386-8ba8-5f18e783013c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:25.018128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130eb80-5543-4386-8ba8-5f18e783013c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.018537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130eb80-5543-4386-8ba8-5f18e783013c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.018813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130eb80-5543-4386-8ba8-5f18e783013c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.019478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130eb80-5543-4386-8ba8-5f18e783013c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:25.021611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130eb80-5543-4386-8ba8-5f18e783013c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:25.048833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:25.057980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1055/2088] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:24 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:25.061619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28d4dd-59de-4cf8-9bde-f312e8ff3e19 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.071741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28d4dd-59de-4cf8-9bde-f312e8ff3e19 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:25.130592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07e652-8bca-4e57-9188-03573d608104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:25.131442 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07e652-8bca-4e57-9188-03573d608104 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.131665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07e652-8bca-4e57-9188-03573d608104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.132142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07e652-8bca-4e57-9188-03573d608104 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.132627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07e652-8bca-4e57-9188-03573d608104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:25.135169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07e652-8bca-4e57-9188-03573d608104 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:25.167553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1034/2089] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:25 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:25.172006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130eb80-5543-4386-8ba8-5f18e783013c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.183059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130eb80-5543-4386-8ba8-5f18e783013c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:25.187956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17778f76-3b4a-4b71-ad63-ad21c0c48445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:25.188633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17778f76-3b4a-4b71-ad63-ad21c0c48445 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.188978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17778f76-3b4a-4b71-ad63-ad21c0c48445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.189318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17778f76-3b4a-4b71-ad63-ad21c0c48445 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.190021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17778f76-3b4a-4b71-ad63-ad21c0c48445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:25.191746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17778f76-3b4a-4b71-ad63-ad21c0c48445 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:25.202104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1056/2090] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:25.205976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07e652-8bca-4e57-9188-03573d608104 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.215904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07e652-8bca-4e57-9188-03573d608104 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:25.221443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e996f-ffc2-4ccd-b0ee-42cdaa276c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:25.222030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e996f-ffc2-4ccd-b0ee-42cdaa276c38 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.222306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e996f-ffc2-4ccd-b0ee-42cdaa276c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.222577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e996f-ffc2-4ccd-b0ee-42cdaa276c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.223254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e996f-ffc2-4ccd-b0ee-42cdaa276c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:25.224821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e996f-ffc2-4ccd-b0ee-42cdaa276c38 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:25.235952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1035/2091] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:25.240052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17778f76-3b4a-4b71-ad63-ad21c0c48445 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.249970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17778f76-3b4a-4b71-ad63-ad21c0c48445 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:25.254535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67149bc2-752c-4e3e-93c8-7f5af0707c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:25.255210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67149bc2-752c-4e3e-93c8-7f5af0707c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.255490 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67149bc2-752c-4e3e-93c8-7f5af0707c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.255760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67149bc2-752c-4e3e-93c8-7f5af0707c1e admin admin] PATH_INFO: `/v3/projects/7fd50ea0f4a04ac397a573250b14a803/users/1c8579a979bf482d895d603491a68748/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.270157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67149bc2-752c-4e3e-93c8-7f5af0707c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd50ea0f4a04ac397a573250b14a803, user_id=1c8579a979bf482d895d603491a68748, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:25.272927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67149bc2-752c-4e3e-93c8-7f5af0707c1e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:25.286612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1057/2092] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:25 2022] PUT /identity/v3/projects/7fd50ea0f4a04ac397a573250b14a803/users/1c8579a979bf482d895d603491a68748/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:25.297688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90a428-6b46-47dc-a07e-46cca45d5b80 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.297980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90a428-6b46-47dc-a07e-46cca45d5b80 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.298262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90a428-6b46-47dc-a07e-46cca45d5b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.325471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7f90a428-6b46-47dc-a07e-46cca45d5b80 None None] MFA Rules not processed for user `1c8579a979bf482d895d603491a68748`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:25.326715 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7f90a428-6b46-47dc-a07e-46cca45d5b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:25.335779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:25.394200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1036/2093] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:25.398663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67149bc2-752c-4e3e-93c8-7f5af0707c1e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.463631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67149bc2-752c-4e3e-93c8-7f5af0707c1e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:25.468988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f77e5-14fd-46af-8747-2bfcb4368906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:25.469592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f77e5-14fd-46af-8747-2bfcb4368906 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.469898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f77e5-14fd-46af-8747-2bfcb4368906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.470170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f77e5-14fd-46af-8747-2bfcb4368906 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.470803 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f77e5-14fd-46af-8747-2bfcb4368906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:25.472740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f77e5-14fd-46af-8747-2bfcb4368906 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:25.498855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1058/2094] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:25 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:25.502923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90a428-6b46-47dc-a07e-46cca45d5b80 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.514225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90a428-6b46-47dc-a07e-46cca45d5b80 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:25.571508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfac36-c1c8-4572-ac9e-541a467646e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:25.572124 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfac36-c1c8-4572-ac9e-541a467646e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.572404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfac36-c1c8-4572-ac9e-541a467646e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.572675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfac36-c1c8-4572-ac9e-541a467646e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.573309 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfac36-c1c8-4572-ac9e-541a467646e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:25.575248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfac36-c1c8-4572-ac9e-541a467646e5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:25.601930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1037/2095] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:25 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:25.603256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f77e5-14fd-46af-8747-2bfcb4368906 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.614730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f77e5-14fd-46af-8747-2bfcb4368906 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:25.620486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64420d97-7a42-409c-965c-4c0b7b3d4832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:25.621405 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64420d97-7a42-409c-965c-4c0b7b3d4832 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.621803 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64420d97-7a42-409c-965c-4c0b7b3d4832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.622065 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64420d97-7a42-409c-965c-4c0b7b3d4832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.623121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64420d97-7a42-409c-965c-4c0b7b3d4832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:25.624548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64420d97-7a42-409c-965c-4c0b7b3d4832 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:25.636056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1059/2096] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:03:25.640559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfac36-c1c8-4572-ac9e-541a467646e5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.651048 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfac36-c1c8-4572-ac9e-541a467646e5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:25.656779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f88459-ff15-4a35-977e-3831c5740dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:25.657440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f88459-ff15-4a35-977e-3831c5740dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.657849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f88459-ff15-4a35-977e-3831c5740dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.658132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f88459-ff15-4a35-977e-3831c5740dd6 admin admin] PATH_INFO: `/v3/projects/ac8d61c20a4f4518b5af0d62ec800c97/users/e73ab1767c74447ca94c4d7702becacf/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.674828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f88459-ff15-4a35-977e-3831c5740dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac8d61c20a4f4518b5af0d62ec800c97, user_id=e73ab1767c74447ca94c4d7702becacf, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:25.677547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f88459-ff15-4a35-977e-3831c5740dd6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:25.693322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1038/2097] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:25 2022] PUT /identity/v3/projects/ac8d61c20a4f4518b5af0d62ec800c97/users/e73ab1767c74447ca94c4d7702becacf/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:25.697317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64420d97-7a42-409c-965c-4c0b7b3d4832 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.709067 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64420d97-7a42-409c-965c-4c0b7b3d4832 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:25.771865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc2628-b5cf-41d3-8e80-de20ce416cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:25.772477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc2628-b5cf-41d3-8e80-de20ce416cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.772753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc2628-b5cf-41d3-8e80-de20ce416cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.773022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc2628-b5cf-41d3-8e80-de20ce416cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.773681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc2628-b5cf-41d3-8e80-de20ce416cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:25.775282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc2628-b5cf-41d3-8e80-de20ce416cbd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:25.786553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1060/2098] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:25.790379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f88459-ff15-4a35-977e-3831c5740dd6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.800983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f88459-ff15-4a35-977e-3831c5740dd6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:25.805719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9fecd-676c-4514-a85a-d94e3c263a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:25.806378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9fecd-676c-4514-a85a-d94e3c263a60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.806656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9fecd-676c-4514-a85a-d94e3c263a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.806933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9fecd-676c-4514-a85a-d94e3c263a60 admin admin] PATH_INFO: `/v3/projects/ac8d61c20a4f4518b5af0d62ec800c97/users/e73ab1767c74447ca94c4d7702becacf/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.809007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9fecd-676c-4514-a85a-d94e3c263a60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac8d61c20a4f4518b5af0d62ec800c97, user_id=e73ab1767c74447ca94c4d7702becacf, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:25.812307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9fecd-676c-4514-a85a-d94e3c263a60 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:25.823679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1039/2099] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:25 2022] PUT /identity/v3/projects/ac8d61c20a4f4518b5af0d62ec800c97/users/e73ab1767c74447ca94c4d7702becacf/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:25.835380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd539800-16d8-4d57-86db-d1a9b2a1a06f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:25.835669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd539800-16d8-4d57-86db-d1a9b2a1a06f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:25.835946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd539800-16d8-4d57-86db-d1a9b2a1a06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:25.864753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fd539800-16d8-4d57-86db-d1a9b2a1a06f None None] MFA Rules not processed for user `e73ab1767c74447ca94c4d7702becacf`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:25.865958 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fd539800-16d8-4d57-86db-d1a9b2a1a06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:25.887530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:25.939915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1061/2100] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:25.952324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9fecd-676c-4514-a85a-d94e3c263a60 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:25.955612 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-60c9fecd-676c-4514-a85a-d94e3c263a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:26.039466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9fecd-676c-4514-a85a-d94e3c263a60 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:26.043077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2163d3-ce91-4935-9d2b-7900815bcd27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:26.043684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2163d3-ce91-4935-9d2b-7900815bcd27 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:26.043963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2163d3-ce91-4935-9d2b-7900815bcd27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:26.044239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2163d3-ce91-4935-9d2b-7900815bcd27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:26.054427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2163d3-ce91-4935-9d2b-7900815bcd27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:26.055886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2163d3-ce91-4935-9d2b-7900815bcd27 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:26.065901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1040/2101] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:03:26.144093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd539800-16d8-4d57-86db-d1a9b2a1a06f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:26.153930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd539800-16d8-4d57-86db-d1a9b2a1a06f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:26.157692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e8b03-5804-4e77-a7eb-112d3391db4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:26.158295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e8b03-5804-4e77-a7eb-112d3391db4f service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:26.158617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e8b03-5804-4e77-a7eb-112d3391db4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:26.158892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e8b03-5804-4e77-a7eb-112d3391db4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:26.246557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e8b03-5804-4e77-a7eb-112d3391db4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:26.249121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e8b03-5804-4e77-a7eb-112d3391db4f service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:26.260236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1062/2102] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:27.061092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2163d3-ce91-4935-9d2b-7900815bcd27 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:27.062697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3a2163d3-ce91-4935-9d2b-7900815bcd27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:27.073545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:27.129636 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2163d3-ce91-4935-9d2b-7900815bcd27 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:27.136041 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:27.136843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:27.137183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:27.137545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] PATH_INFO: `/v3/users/e2994c9e4bc24406aec8824b81f6ea0c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:27.139415 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2994c9e4bc24406aec8824b81f6ea0c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:27.141338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:27.186674 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2994c9e4bc24406aec8824b81f6ea0c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.198466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2994c9e4bc24406aec8824b81f6ea0c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.199086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2994c9e4bc24406aec8824b81f6ea0c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.199686 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2994c9e4bc24406aec8824b81f6ea0c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.207378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2994c9e4bc24406aec8824b81f6ea0c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.220548 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1041/2103] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:27 2022] DELETE /identity/v3/users/e2994c9e4bc24406aec8824b81f6ea0c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:27.396048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e8b03-5804-4e77-a7eb-112d3391db4f service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:27.407757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e8b03-5804-4e77-a7eb-112d3391db4f service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:27.467286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:27.467913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:27.468188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:27.468462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] PATH_INFO: `/v3/projects/d64f9d1b73c24a21a4c49565d436836e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:27.469572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64f9d1b73c24a21a4c49565d436836e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:27.471530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:27.539106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64f9d1b73c24a21a4c49565d436836e', 'request_id': 'req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.552916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64f9d1b73c24a21a4c49565d436836e', 'request_id': 'req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.555229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:27.561477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64f9d1b73c24a21a4c49565d436836e', 'request_id': 'req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.580622 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] The token cache is being invalidate because project d64f9d1b73c24a21a4c49565d436836e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:27.581153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.583543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.585498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1063/2104] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:27 2022] DELETE /identity/v3/projects/d64f9d1b73c24a21a4c49565d436836e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:27.589749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:27.591350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:27.607772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ab459-ce13-4bc3-bd97-ea9ae9d68d3d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:27.699752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963816c-6617-4e9c-b1e4-e59cb707058b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:27.700473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963816c-6617-4e9c-b1e4-e59cb707058b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:27.701270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963816c-6617-4e9c-b1e4-e59cb707058b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:27.701624 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963816c-6617-4e9c-b1e4-e59cb707058b admin admin] PATH_INFO: `/v3/users/3b5137ff267e4566ab310723c4757014` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:27.702814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963816c-6617-4e9c-b1e4-e59cb707058b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b5137ff267e4566ab310723c4757014)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:27.705259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963816c-6617-4e9c-b1e4-e59cb707058b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:27.750391 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:27.771342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5963816c-6617-4e9c-b1e4-e59cb707058b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b5137ff267e4566ab310723c4757014'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.783702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5963816c-6617-4e9c-b1e4-e59cb707058b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b5137ff267e4566ab310723c4757014'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.784460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5963816c-6617-4e9c-b1e4-e59cb707058b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b5137ff267e4566ab310723c4757014'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.785304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5963816c-6617-4e9c-b1e4-e59cb707058b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b5137ff267e4566ab310723c4757014'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.792710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5963816c-6617-4e9c-b1e4-e59cb707058b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b5137ff267e4566ab310723c4757014'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:27.806735 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1042/2105] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:27 2022] DELETE /identity/v3/users/3b5137ff267e4566ab310723c4757014 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:28.006710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:28.008217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:28.021961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a058e-49a2-4c2c-8a2a-795237ecb0c6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:28.083861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:28.084657 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.084935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.085201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] PATH_INFO: `/v3/projects/4a32cd6bf5264316bb70d9ae94e87876` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.086319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a32cd6bf5264316bb70d9ae94e87876)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:28.088240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:28.137489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a32cd6bf5264316bb70d9ae94e87876', 'request_id': 'req-ac3bc375-38f8-4948-8c54-17a5ef3ae888'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:28.144859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a32cd6bf5264316bb70d9ae94e87876', 'request_id': 'req-ac3bc375-38f8-4948-8c54-17a5ef3ae888'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:28.151925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a32cd6bf5264316bb70d9ae94e87876', 'request_id': 'req-ac3bc375-38f8-4948-8c54-17a5ef3ae888'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:28.163314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] The token cache is being invalidate because project 4a32cd6bf5264316bb70d9ae94e87876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:28.163607 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:28.164259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac3bc375-38f8-4948-8c54-17a5ef3ae888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:28.165874 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1064/2106] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:28 2022] DELETE /identity/v3/projects/4a32cd6bf5264316bb70d9ae94e87876 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:28.171378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8a3de-8160-4eeb-af10-13c8bc231d20 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.171831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8a3de-8160-4eeb-af10-13c8bc231d20 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.171950 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8a3de-8160-4eeb-af10-13c8bc231d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.191474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-54b8a3de-8160-4eeb-af10-13c8bc231d20 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:28.192666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-54b8a3de-8160-4eeb-af10-13c8bc231d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:28.250856 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1043/2107] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:28.258652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41260618-8f0e-47cf-a68e-7d018db052f4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.258946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41260618-8f0e-47cf-a68e-7d018db052f4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.259288 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41260618-8f0e-47cf-a68e-7d018db052f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.280346 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-41260618-8f0e-47cf-a68e-7d018db052f4 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:28.281550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-41260618-8f0e-47cf-a68e-7d018db052f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:28.289483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1065/2108] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:28.293726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8a3de-8160-4eeb-af10-13c8bc231d20 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:28.304193 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8a3de-8160-4eeb-af10-13c8bc231d20 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:28.308699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958265a-fb7e-4f4e-8bd7-3e96fdfe3ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:28.309292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958265a-fb7e-4f4e-8bd7-3e96fdfe3ecd admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.309568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958265a-fb7e-4f4e-8bd7-3e96fdfe3ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.309839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958265a-fb7e-4f4e-8bd7-3e96fdfe3ecd admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.310473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958265a-fb7e-4f4e-8bd7-3e96fdfe3ecd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:03:28.310772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958265a-fb7e-4f4e-8bd7-3e96fdfe3ecd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:28.312381 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958265a-fb7e-4f4e-8bd7-3e96fdfe3ecd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:28.324036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1044/2109] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:03:28.328438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41260618-8f0e-47cf-a68e-7d018db052f4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:28.337796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41260618-8f0e-47cf-a68e-7d018db052f4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:28.343376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7100cb2-c082-42c3-8b0e-eca86e1d949f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:28.343966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7100cb2-c082-42c3-8b0e-eca86e1d949f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.346558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7100cb2-c082-42c3-8b0e-eca86e1d949f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.346558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7100cb2-c082-42c3-8b0e-eca86e1d949f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.346558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7100cb2-c082-42c3-8b0e-eca86e1d949f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:28.347170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7100cb2-c082-42c3-8b0e-eca86e1d949f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:28.375988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1066/2110] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:28 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:28.380401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958265a-fb7e-4f4e-8bd7-3e96fdfe3ecd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:28.391079 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958265a-fb7e-4f4e-8bd7-3e96fdfe3ecd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:28.450588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f4ec2-8246-4cea-9754-66178bd3742c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:28.451315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f4ec2-8246-4cea-9754-66178bd3742c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.451574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f4ec2-8246-4cea-9754-66178bd3742c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.451841 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f4ec2-8246-4cea-9754-66178bd3742c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.452468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f4ec2-8246-4cea-9754-66178bd3742c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:28.454410 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f4ec2-8246-4cea-9754-66178bd3742c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:28.475176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1045/2111] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:28 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:28.479313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7100cb2-c082-42c3-8b0e-eca86e1d949f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:28.490166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7100cb2-c082-42c3-8b0e-eca86e1d949f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:28.494753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168c0bf-5c64-4286-a70b-741f0761bf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:28.495381 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168c0bf-5c64-4286-a70b-741f0761bf64 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.495672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168c0bf-5c64-4286-a70b-741f0761bf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.495960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168c0bf-5c64-4286-a70b-741f0761bf64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.496591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168c0bf-5c64-4286-a70b-741f0761bf64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:28.498182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168c0bf-5c64-4286-a70b-741f0761bf64 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:28.509458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1067/2112] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:28.513287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f4ec2-8246-4cea-9754-66178bd3742c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:28.523470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f4ec2-8246-4cea-9754-66178bd3742c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:28.528173 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ff4ef-278b-4cc9-819b-be4e2a8f9a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:28.528760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ff4ef-278b-4cc9-819b-be4e2a8f9a9f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.529038 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ff4ef-278b-4cc9-819b-be4e2a8f9a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.529798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ff4ef-278b-4cc9-819b-be4e2a8f9a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.530432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ff4ef-278b-4cc9-819b-be4e2a8f9a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:28.532076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ff4ef-278b-4cc9-819b-be4e2a8f9a9f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:28.543307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1046/2113] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:28.548363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168c0bf-5c64-4286-a70b-741f0761bf64 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:28.558326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168c0bf-5c64-4286-a70b-741f0761bf64 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:28.562982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b418c-e4c4-4fa5-9d42-af790590a841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:28.563649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b418c-e4c4-4fa5-9d42-af790590a841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.563927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b418c-e4c4-4fa5-9d42-af790590a841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.564197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b418c-e4c4-4fa5-9d42-af790590a841 admin admin] PATH_INFO: `/v3/projects/91ea292631c3433699fa30b052e73b4f/users/6baa720eb4504e44a5a4c5cb57d0ed73/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.578506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b418c-e4c4-4fa5-9d42-af790590a841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ea292631c3433699fa30b052e73b4f, user_id=6baa720eb4504e44a5a4c5cb57d0ed73, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:28.581264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b418c-e4c4-4fa5-9d42-af790590a841 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:28.594189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1068/2114] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:28 2022] PUT /identity/v3/projects/91ea292631c3433699fa30b052e73b4f/users/6baa720eb4504e44a5a4c5cb57d0ed73/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:28.604432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b65e8-5897-4af5-8c70-67abb91e2fcf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.604746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b65e8-5897-4af5-8c70-67abb91e2fcf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.605047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b65e8-5897-4af5-8c70-67abb91e2fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.627651 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:28.637950 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-601b65e8-5897-4af5-8c70-67abb91e2fcf None None] MFA Rules not processed for user `6baa720eb4504e44a5a4c5cb57d0ed73`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:28.639600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-601b65e8-5897-4af5-8c70-67abb91e2fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:28.737620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1047/2115] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:28.740062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b418c-e4c4-4fa5-9d42-af790590a841 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:28.761348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:28.834169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b418c-e4c4-4fa5-9d42-af790590a841 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:28.839421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f4f1a-b871-4b72-8958-9b514d893510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:28.840120 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f4f1a-b871-4b72-8958-9b514d893510 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.840465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f4f1a-b871-4b72-8958-9b514d893510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.840801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f4f1a-b871-4b72-8958-9b514d893510 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.842472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f4f1a-b871-4b72-8958-9b514d893510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:28.844624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f4f1a-b871-4b72-8958-9b514d893510 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:28.877277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1069/2116] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:28 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:28.881128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b65e8-5897-4af5-8c70-67abb91e2fcf None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:28.893672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b65e8-5897-4af5-8c70-67abb91e2fcf None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:28.963640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4e359-fa37-43f5-920c-6a6b58ac5713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:28.966351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4e359-fa37-43f5-920c-6a6b58ac5713 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:28.966351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4e359-fa37-43f5-920c-6a6b58ac5713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:28.966351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4e359-fa37-43f5-920c-6a6b58ac5713 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:28.968012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4e359-fa37-43f5-920c-6a6b58ac5713 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:28.970284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4e359-fa37-43f5-920c-6a6b58ac5713 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:29.017738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f4f1a-b871-4b72-8958-9b514d893510 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:29.019235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1048/2117] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:28 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:03:29.039824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f4f1a-b871-4b72-8958-9b514d893510 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:29.053050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b566e7-6b71-45d1-a47a-8e51867bc45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:29.054629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b566e7-6b71-45d1-a47a-8e51867bc45f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:29.055096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b566e7-6b71-45d1-a47a-8e51867bc45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:29.055467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b566e7-6b71-45d1-a47a-8e51867bc45f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:29.056498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b566e7-6b71-45d1-a47a-8e51867bc45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:29.058951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b566e7-6b71-45d1-a47a-8e51867bc45f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:29.075309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1070/2118] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:29.081298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4e359-fa37-43f5-920c-6a6b58ac5713 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:29.101181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4e359-fa37-43f5-920c-6a6b58ac5713 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:29.111648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbf374-2f6f-47f7-976d-db3a41a87cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:29.112922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbf374-2f6f-47f7-976d-db3a41a87cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:29.113540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbf374-2f6f-47f7-976d-db3a41a87cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:29.113918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbf374-2f6f-47f7-976d-db3a41a87cbc admin admin] PATH_INFO: `/v3/projects/96a0839fc1664049bc85133be46165ed/users/c2041104ac3e4394b79cd7045b0c6633/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:29.143088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbf374-2f6f-47f7-976d-db3a41a87cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a0839fc1664049bc85133be46165ed, user_id=c2041104ac3e4394b79cd7045b0c6633, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:29.148923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbf374-2f6f-47f7-976d-db3a41a87cbc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:29.170879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1049/2119] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:29 2022] PUT /identity/v3/projects/96a0839fc1664049bc85133be46165ed/users/c2041104ac3e4394b79cd7045b0c6633/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:29.172099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b566e7-6b71-45d1-a47a-8e51867bc45f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:29.185133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b566e7-6b71-45d1-a47a-8e51867bc45f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:29.311736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd41f4d-b3ef-4db7-9704-9e056e7f0ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:29.312686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd41f4d-b3ef-4db7-9704-9e056e7f0ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:29.313588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd41f4d-b3ef-4db7-9704-9e056e7f0ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:29.314187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd41f4d-b3ef-4db7-9704-9e056e7f0ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:29.315095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd41f4d-b3ef-4db7-9704-9e056e7f0ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:29.317438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd41f4d-b3ef-4db7-9704-9e056e7f0ec8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:29.332294 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1071/2120] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:29.336380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbf374-2f6f-47f7-976d-db3a41a87cbc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:29.350470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbf374-2f6f-47f7-976d-db3a41a87cbc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:29.356095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6ddb1-aa14-408c-a3f5-b55dd6f83b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:29.356873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ddb1-aa14-408c-a3f5-b55dd6f83b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:29.357216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ddb1-aa14-408c-a3f5-b55dd6f83b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:29.357590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ddb1-aa14-408c-a3f5-b55dd6f83b51 admin admin] PATH_INFO: `/v3/projects/96a0839fc1664049bc85133be46165ed/users/c2041104ac3e4394b79cd7045b0c6633/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:29.359936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6ddb1-aa14-408c-a3f5-b55dd6f83b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a0839fc1664049bc85133be46165ed, user_id=c2041104ac3e4394b79cd7045b0c6633, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:29.362949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6ddb1-aa14-408c-a3f5-b55dd6f83b51 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:29.394185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1050/2121] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:29 2022] PUT /identity/v3/projects/96a0839fc1664049bc85133be46165ed/users/c2041104ac3e4394b79cd7045b0c6633/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:29.406710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333b998-f436-401f-9266-db8da535dee2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:29.407033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333b998-f436-401f-9266-db8da535dee2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:29.407403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333b998-f436-401f-9266-db8da535dee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:29.457960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d333b998-f436-401f-9266-db8da535dee2 None None] MFA Rules not processed for user `c2041104ac3e4394b79cd7045b0c6633`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:29.457960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d333b998-f436-401f-9266-db8da535dee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:29.533969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1072/2122] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:29.545466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6ddb1-aa14-408c-a3f5-b55dd6f83b51 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:29.547308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-54e6ddb1-aa14-408c-a3f5-b55dd6f83b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:29.608418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6ddb1-aa14-408c-a3f5-b55dd6f83b51 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:29.612384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4267d5-c4fa-4c75-8a85-a3017d6146e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:29.612999 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4267d5-c4fa-4c75-8a85-a3017d6146e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:29.613280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4267d5-c4fa-4c75-8a85-a3017d6146e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:29.613549 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4267d5-c4fa-4c75-8a85-a3017d6146e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:29.619281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:29.681469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4267d5-c4fa-4c75-8a85-a3017d6146e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:29.682927 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4267d5-c4fa-4c75-8a85-a3017d6146e3 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:29.695082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1051/2123] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:36.350875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f2470-c5ea-49ad-a4c9-e7792f9d706c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:36.351578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f2470-c5ea-49ad-a4c9-e7792f9d706c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:36.351578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f2470-c5ea-49ad-a4c9-e7792f9d706c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:36.370816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7a1f2470-c5ea-49ad-a4c9-e7792f9d706c None None] MFA Rules not processed for user `41227378a57c43219e4ae460c642f834`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:36.372043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7a1f2470-c5ea-49ad-a4c9-e7792f9d706c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:36.376150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:36.427904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1073/2124] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:38.694741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4267d5-c4fa-4c75-8a85-a3017d6146e3 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:38.696745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0e4267d5-c4fa-4c75-8a85-a3017d6146e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:38.798095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4267d5-c4fa-4c75-8a85-a3017d6146e3 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:38.802667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:38.803343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:38.803757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:38.804041 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] PATH_INFO: `/v3/users/1c8579a979bf482d895d603491a68748` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:38.805383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8579a979bf482d895d603491a68748)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:38.807302 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:38.900616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8579a979bf482d895d603491a68748'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:38.913582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8579a979bf482d895d603491a68748'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:38.914303 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8579a979bf482d895d603491a68748'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:38.914918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8579a979bf482d895d603491a68748'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:38.924356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8579a979bf482d895d603491a68748'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:38.940805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1052/2125] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:38 2022] DELETE /identity/v3/users/1c8579a979bf482d895d603491a68748 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:38.945758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1541624-71fa-49c4-bb32-d19d26e22ba4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:38.946251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1541624-71fa-49c4-bb32-d19d26e22ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:38.946344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1541624-71fa-49c4-bb32-d19d26e22ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:38.966266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d1541624-71fa-49c4-bb32-d19d26e22ba4 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:38.968096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d1541624-71fa-49c4-bb32-d19d26e22ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:39.041644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1074/2126] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:39.062012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:39.063661 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:39.099300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:39.141976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77955027-6b55-43d8-832a-ceaf24561397 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:39.146345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36f728-b72a-46c4-b673-214e8275997f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:39.146976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36f728-b72a-46c4-b673-214e8275997f service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:39.147289 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36f728-b72a-46c4-b673-214e8275997f service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:39.147565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36f728-b72a-46c4-b673-214e8275997f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:39.160075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36f728-b72a-46c4-b673-214e8275997f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:39.161655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36f728-b72a-46c4-b673-214e8275997f service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:39.173523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1053/2127] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:39.483668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1541624-71fa-49c4-bb32-d19d26e22ba4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:39.486417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d1541624-71fa-49c4-bb32-d19d26e22ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:39.569115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1541624-71fa-49c4-bb32-d19d26e22ba4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:39.574707 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e78622-9ad7-400c-9e04-87258c38ccca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:39.575639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e78622-9ad7-400c-9e04-87258c38ccca service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:39.576480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e78622-9ad7-400c-9e04-87258c38ccca service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:39.576934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e78622-9ad7-400c-9e04-87258c38ccca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:39.579196 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a9e78622-9ad7-400c-9e04-87258c38ccca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:39.675161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e78622-9ad7-400c-9e04-87258c38ccca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:39.678031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e78622-9ad7-400c-9e04-87258c38ccca service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:39.689774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1075/2128] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:40.065357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36f728-b72a-46c4-b673-214e8275997f service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:40.067472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9f36f728-b72a-46c4-b673-214e8275997f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:40.091535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36f728-b72a-46c4-b673-214e8275997f service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:40.102936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:40.103723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:40.104130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:40.104470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] PATH_INFO: `/v3/projects/7fd50ea0f4a04ac397a573250b14a803` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:40.105833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fd50ea0f4a04ac397a573250b14a803)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:40.108256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:40.189299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fd50ea0f4a04ac397a573250b14a803', 'request_id': 'req-af6517b3-17cc-4e17-806d-57f68a4f27a7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.197496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fd50ea0f4a04ac397a573250b14a803', 'request_id': 'req-af6517b3-17cc-4e17-806d-57f68a4f27a7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.209001 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fd50ea0f4a04ac397a573250b14a803', 'request_id': 'req-af6517b3-17cc-4e17-806d-57f68a4f27a7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.242782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] The token cache is being invalidate because project 7fd50ea0f4a04ac397a573250b14a803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:40.243113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.243777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.245241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1054/2129] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:40 2022] DELETE /identity/v3/projects/7fd50ea0f4a04ac397a573250b14a803 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:40.249447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e78622-9ad7-400c-9e04-87258c38ccca service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:40.277796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:40.285080 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e78622-9ad7-400c-9e04-87258c38ccca service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:40.370884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58744856-eabd-4f98-805d-52919700bb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:40.371564 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58744856-eabd-4f98-805d-52919700bb20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:40.371852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58744856-eabd-4f98-805d-52919700bb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:40.372121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58744856-eabd-4f98-805d-52919700bb20 admin admin] PATH_INFO: `/v3/users/e73ab1767c74447ca94c4d7702becacf` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:40.373985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58744856-eabd-4f98-805d-52919700bb20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e73ab1767c74447ca94c4d7702becacf)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:40.375997 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58744856-eabd-4f98-805d-52919700bb20 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:40.447117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-58744856-eabd-4f98-805d-52919700bb20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e73ab1767c74447ca94c4d7702becacf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.447809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-58744856-eabd-4f98-805d-52919700bb20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e73ab1767c74447ca94c4d7702becacf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.471953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-58744856-eabd-4f98-805d-52919700bb20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e73ab1767c74447ca94c4d7702becacf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.481123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-58744856-eabd-4f98-805d-52919700bb20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e73ab1767c74447ca94c4d7702becacf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.488560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-58744856-eabd-4f98-805d-52919700bb20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e73ab1767c74447ca94c4d7702becacf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.495513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1076/2130] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:40 2022] DELETE /identity/v3/users/e73ab1767c74447ca94c4d7702becacf => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:40.793008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:40.795128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:40.815686 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6517b3-17cc-4e17-806d-57f68a4f27a7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:40.888065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:40.889226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:40.889878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:40.889878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] PATH_INFO: `/v3/projects/ac8d61c20a4f4518b5af0d62ec800c97` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:40.890930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac8d61c20a4f4518b5af0d62ec800c97)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:40.892882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:40.952663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac8d61c20a4f4518b5af0d62ec800c97', 'request_id': 'req-84201869-9cbe-43e8-acfd-d180679388ba'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.959757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac8d61c20a4f4518b5af0d62ec800c97', 'request_id': 'req-84201869-9cbe-43e8-acfd-d180679388ba'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.961433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:40.970485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac8d61c20a4f4518b5af0d62ec800c97', 'request_id': 'req-84201869-9cbe-43e8-acfd-d180679388ba'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.985744 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] The token cache is being invalidate because project ac8d61c20a4f4518b5af0d62ec800c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:40.986180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.987094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-84201869-9cbe-43e8-acfd-d180679388ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:40.989211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1055/2131] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:40 2022] DELETE /identity/v3/projects/ac8d61c20a4f4518b5af0d62ec800c97 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:40.994255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3356d4f-b323-488b-9e26-5592b460f0f3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:40.994546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3356d4f-b323-488b-9e26-5592b460f0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:40.994825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3356d4f-b323-488b-9e26-5592b460f0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:41.015296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e3356d4f-b323-488b-9e26-5592b460f0f3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:41.016534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e3356d4f-b323-488b-9e26-5592b460f0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:41.119407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1077/2132] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:03:41.127054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1de02-1142-402c-87cc-204548de02d4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:41.127588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1de02-1142-402c-87cc-204548de02d4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:41.127669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1de02-1142-402c-87cc-204548de02d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:41.180792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-76f1de02-1142-402c-87cc-204548de02d4 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:41.182476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-76f1de02-1142-402c-87cc-204548de02d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:41.201568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1056/2133] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:41.208396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3356d4f-b323-488b-9e26-5592b460f0f3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:41.222024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3356d4f-b323-488b-9e26-5592b460f0f3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:41.229357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c13176-f7b3-402f-a49a-73233b6c46ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:41.230313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c13176-f7b3-402f-a49a-73233b6c46ba admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:41.230744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c13176-f7b3-402f-a49a-73233b6c46ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:41.231240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c13176-f7b3-402f-a49a-73233b6c46ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:41.232514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c13176-f7b3-402f-a49a-73233b6c46ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:03:41.233006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c13176-f7b3-402f-a49a-73233b6c46ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:41.235375 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c13176-f7b3-402f-a49a-73233b6c46ba admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:41.254468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1078/2134] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:03:41.259040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1de02-1142-402c-87cc-204548de02d4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:41.273520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1de02-1142-402c-87cc-204548de02d4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:41.297623 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641ea70-0181-47a8-b847-6572c6e5df61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:41.301549 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641ea70-0181-47a8-b847-6572c6e5df61 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:41.301943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641ea70-0181-47a8-b847-6572c6e5df61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:41.302221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641ea70-0181-47a8-b847-6572c6e5df61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:41.303780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641ea70-0181-47a8-b847-6572c6e5df61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:41.306022 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641ea70-0181-47a8-b847-6572c6e5df61 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:41.368908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1057/2135] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:41 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:41.370213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c13176-f7b3-402f-a49a-73233b6c46ba admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:41.394948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c13176-f7b3-402f-a49a-73233b6c46ba admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:41.411826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:41.506447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bb47-e15f-4b63-ad6b-dcc6a66c9654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:41.507118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bb47-e15f-4b63-ad6b-dcc6a66c9654 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:41.507673 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bb47-e15f-4b63-ad6b-dcc6a66c9654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:41.507973 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bb47-e15f-4b63-ad6b-dcc6a66c9654 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:41.508616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455bb47-e15f-4b63-ad6b-dcc6a66c9654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:41.510833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455bb47-e15f-4b63-ad6b-dcc6a66c9654 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:41.560950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1079/2136] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:41 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:41.562534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641ea70-0181-47a8-b847-6572c6e5df61 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:41.577181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641ea70-0181-47a8-b847-6572c6e5df61 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:41.584585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a1b8b-772e-4d93-8ea3-13f2a2e05ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:41.585277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a1b8b-772e-4d93-8ea3-13f2a2e05ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:41.585738 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a1b8b-772e-4d93-8ea3-13f2a2e05ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:41.585941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a1b8b-772e-4d93-8ea3-13f2a2e05ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:41.586870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a1b8b-772e-4d93-8ea3-13f2a2e05ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:41.588892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a1b8b-772e-4d93-8ea3-13f2a2e05ffa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:41.606458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1058/2137] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:03:41.608422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bb47-e15f-4b63-ad6b-dcc6a66c9654 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:41.636484 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bb47-e15f-4b63-ad6b-dcc6a66c9654 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:41.659846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbf5d8-8a17-4101-b816-46bbfe7edd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:41.660625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbf5d8-8a17-4101-b816-46bbfe7edd37 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:41.660999 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbf5d8-8a17-4101-b816-46bbfe7edd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:41.661462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbf5d8-8a17-4101-b816-46bbfe7edd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:41.662113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbf5d8-8a17-4101-b816-46bbfe7edd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:41.664784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbf5d8-8a17-4101-b816-46bbfe7edd37 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:41.704905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1080/2138] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:41.708907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a1b8b-772e-4d93-8ea3-13f2a2e05ffa admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:41.737479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a1b8b-772e-4d93-8ea3-13f2a2e05ffa admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:41.746294 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f769d-96b0-4106-b1b3-77b1085ed40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:41.747089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f769d-96b0-4106-b1b3-77b1085ed40d admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:41.747430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f769d-96b0-4106-b1b3-77b1085ed40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:41.747860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f769d-96b0-4106-b1b3-77b1085ed40d admin admin] PATH_INFO: `/v3/projects/9543c8801b5b499682005c2f254208be/users/10e43105015d477cb940eb3a6392090f/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:41.765619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f769d-96b0-4106-b1b3-77b1085ed40d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9543c8801b5b499682005c2f254208be, user_id=10e43105015d477cb940eb3a6392090f, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:41.768547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f769d-96b0-4106-b1b3-77b1085ed40d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:41.798342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1059/2139] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:41 2022] PUT /identity/v3/projects/9543c8801b5b499682005c2f254208be/users/10e43105015d477cb940eb3a6392090f/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:41.809983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7ebc5-c499-4d70-b141-686bece48449 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:41.810666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7ebc5-c499-4d70-b141-686bece48449 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:41.811098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7ebc5-c499-4d70-b141-686bece48449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:41.855043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2fa7ebc5-c499-4d70-b141-686bece48449 None None] MFA Rules not processed for user `10e43105015d477cb940eb3a6392090f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:41.856507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2fa7ebc5-c499-4d70-b141-686bece48449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:41.945109 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1081/2140] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:41.946776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f769d-96b0-4106-b1b3-77b1085ed40d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:42.019112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7ebc5-c499-4d70-b141-686bece48449 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:42.020998 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2fa7ebc5-c499-4d70-b141-686bece48449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:42.034244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f769d-96b0-4106-b1b3-77b1085ed40d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:42.040143 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d436c-2746-4600-bdfc-8e9a9168f04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:42.040786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d436c-2746-4600-bdfc-8e9a9168f04b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.041074 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d436c-2746-4600-bdfc-8e9a9168f04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.041360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d436c-2746-4600-bdfc-8e9a9168f04b admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.041997 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d436c-2746-4600-bdfc-8e9a9168f04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:42.044840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d436c-2746-4600-bdfc-8e9a9168f04b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:42.109435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7ebc5-c499-4d70-b141-686bece48449 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:42.119143 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1060/2141] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:41 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:42.120365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d436c-2746-4600-bdfc-8e9a9168f04b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:42.147645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:42.192452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:42.193370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.193651 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.193923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] PATH_INFO: `/v3/users/5598718e51df4a7c92ef828de3003dc7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.195143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5598718e51df4a7c92ef828de3003dc7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:42.197378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:42.207727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d436c-2746-4600-bdfc-8e9a9168f04b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:42.213162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9063a-9803-468d-8ddf-99317402ae50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:42.213473 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9063a-9803-468d-8ddf-99317402ae50 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.213727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9063a-9803-468d-8ddf-99317402ae50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.214015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9063a-9803-468d-8ddf-99317402ae50 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.214711 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b9063a-9803-468d-8ddf-99317402ae50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:42.217360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b9063a-9803-468d-8ddf-99317402ae50 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:42.228187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=3,keystone:SELECT=91,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:42.245324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1061/2142] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:42 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:42.252450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9063a-9803-468d-8ddf-99317402ae50 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:42.260162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5598718e51df4a7c92ef828de3003dc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:42.260863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5598718e51df4a7c92ef828de3003dc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:42.264365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9063a-9803-468d-8ddf-99317402ae50 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:42.269327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111911f6-f4eb-4aa9-8980-78b02331c354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:42.269936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111911f6-f4eb-4aa9-8980-78b02331c354 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.270218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111911f6-f4eb-4aa9-8980-78b02331c354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.270495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111911f6-f4eb-4aa9-8980-78b02331c354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.271203 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111911f6-f4eb-4aa9-8980-78b02331c354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:42.272805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111911f6-f4eb-4aa9-8980-78b02331c354 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:42.275647 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5598718e51df4a7c92ef828de3003dc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:42.284944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5598718e51df4a7c92ef828de3003dc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:42.285241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1062/2143] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:42.288817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111911f6-f4eb-4aa9-8980-78b02331c354 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:42.290521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-111911f6-f4eb-4aa9-8980-78b02331c354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:42.292859 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5598718e51df4a7c92ef828de3003dc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:42.304757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111911f6-f4eb-4aa9-8980-78b02331c354 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:42.307062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1082/2144] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:42 2022] DELETE /identity/v3/users/5598718e51df4a7c92ef828de3003dc7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:42.396407 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f19630-fe1a-4620-abe0-fe2177a11195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:42.397447 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f19630-fe1a-4620-abe0-fe2177a11195 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.397879 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f19630-fe1a-4620-abe0-fe2177a11195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.398310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f19630-fe1a-4620-abe0-fe2177a11195 admin admin] PATH_INFO: `/v3/projects/26c698be17d3418b8d5a93ee0e8251c3/users/ac2240be1a4749a1a66f6544a0ba949e/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.416858 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f19630-fe1a-4620-abe0-fe2177a11195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26c698be17d3418b8d5a93ee0e8251c3, user_id=ac2240be1a4749a1a66f6544a0ba949e, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:42.420668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f19630-fe1a-4620-abe0-fe2177a11195 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:42.438777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1063/2145] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:42 2022] PUT /identity/v3/projects/26c698be17d3418b8d5a93ee0e8251c3/users/ac2240be1a4749a1a66f6544a0ba949e/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:42.442792 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:42.454879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049650a-d596-4f9b-98e3-b89ce2495e61 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:42.514043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f19630-fe1a-4620-abe0-fe2177a11195 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:42.516505 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-39f19630-fe1a-4620-abe0-fe2177a11195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:42.534167 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c249534-e473-4e93-b3ab-e9534f9a71d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:42.534869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c249534-e473-4e93-b3ab-e9534f9a71d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.535485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c249534-e473-4e93-b3ab-e9534f9a71d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.535807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c249534-e473-4e93-b3ab-e9534f9a71d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.536514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c249534-e473-4e93-b3ab-e9534f9a71d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:42.539520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c249534-e473-4e93-b3ab-e9534f9a71d6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:42.558222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1083/2146] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:42.562973 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c249534-e473-4e93-b3ab-e9534f9a71d6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:42.577461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c249534-e473-4e93-b3ab-e9534f9a71d6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:42.584778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a242a-530c-4e36-8b02-a76442cdcb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:42.585494 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a242a-530c-4e36-8b02-a76442cdcb34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.585844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a242a-530c-4e36-8b02-a76442cdcb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.586532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a242a-530c-4e36-8b02-a76442cdcb34 admin admin] PATH_INFO: `/v3/projects/26c698be17d3418b8d5a93ee0e8251c3/users/ac2240be1a4749a1a66f6544a0ba949e/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.589075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a242a-530c-4e36-8b02-a76442cdcb34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26c698be17d3418b8d5a93ee0e8251c3, user_id=ac2240be1a4749a1a66f6544a0ba949e, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:42.592786 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a242a-530c-4e36-8b02-a76442cdcb34 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:42.604213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f19630-fe1a-4620-abe0-fe2177a11195 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:42.605456 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1084/2147] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:42 2022] PUT /identity/v3/projects/26c698be17d3418b8d5a93ee0e8251c3/users/ac2240be1a4749a1a66f6544a0ba949e/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:42.616876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63a67e-7703-4295-9e26-900ff7c99bd6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.617171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63a67e-7703-4295-9e26-900ff7c99bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.617469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63a67e-7703-4295-9e26-900ff7c99bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.650913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8c63a67e-7703-4295-9e26-900ff7c99bd6 None None] MFA Rules not processed for user `ac2240be1a4749a1a66f6544a0ba949e`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:42.652316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8c63a67e-7703-4295-9e26-900ff7c99bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:42.681440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:42.704093 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:42.704855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.705166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.705493 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] PATH_INFO: `/v3/projects/4c6272dcd88f47abb7a514070aaaccd3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.706782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6272dcd88f47abb7a514070aaaccd3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:42.709238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:42.728769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:42.749702 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1085/2148] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:42.760691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63a67e-7703-4295-9e26-900ff7c99bd6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:42.762059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8c63a67e-7703-4295-9e26-900ff7c99bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:42.776356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6272dcd88f47abb7a514070aaaccd3', 'request_id': 'req-826ee44d-1df1-4940-a2d8-ac08a385b57d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:42.787551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6272dcd88f47abb7a514070aaaccd3', 'request_id': 'req-826ee44d-1df1-4940-a2d8-ac08a385b57d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:42.795388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6272dcd88f47abb7a514070aaaccd3', 'request_id': 'req-826ee44d-1df1-4940-a2d8-ac08a385b57d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:42.812068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] The token cache is being invalidate because project 4c6272dcd88f47abb7a514070aaaccd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:42.812368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:42.813731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826ee44d-1df1-4940-a2d8-ac08a385b57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:42.816483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1064/2149] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:42 2022] DELETE /identity/v3/projects/4c6272dcd88f47abb7a514070aaaccd3 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:42.821491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80846e7c-ce30-4607-8f52-309282c3f65c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.822383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80846e7c-ce30-4607-8f52-309282c3f65c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.822645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80846e7c-ce30-4607-8f52-309282c3f65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.837146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63a67e-7703-4295-9e26-900ff7c99bd6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:42.845173 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-80846e7c-ce30-4607-8f52-309282c3f65c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:42.846392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-80846e7c-ce30-4607-8f52-309282c3f65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:42.893764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225eb41d-40ee-46ca-9aec-d3ce1e75f023 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:42.894253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225eb41d-40ee-46ca-9aec-d3ce1e75f023 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.894531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225eb41d-40ee-46ca-9aec-d3ce1e75f023 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.894932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225eb41d-40ee-46ca-9aec-d3ce1e75f023 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.896730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-225eb41d-40ee-46ca-9aec-d3ce1e75f023 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:42.910559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1065/2150] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:42.918223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc202b-3e20-45d7-be7e-6af74aca932a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.918506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc202b-3e20-45d7-be7e-6af74aca932a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.918782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc202b-3e20-45d7-be7e-6af74aca932a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.939168 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-51dc202b-3e20-45d7-be7e-6af74aca932a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:42.940399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-51dc202b-3e20-45d7-be7e-6af74aca932a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:42.948729 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1066/2151] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:03:42.952885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc202b-3e20-45d7-be7e-6af74aca932a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:42.962919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc202b-3e20-45d7-be7e-6af74aca932a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:42.965820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225eb41d-40ee-46ca-9aec-d3ce1e75f023 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:42.968183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225eb41d-40ee-46ca-9aec-d3ce1e75f023 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:42.969376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed589d-9b2f-4a38-9780-25b50a4c62b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:42.969991 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed589d-9b2f-4a38-9780-25b50a4c62b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:42.970272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed589d-9b2f-4a38-9780-25b50a4c62b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:42.970538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed589d-9b2f-4a38-9780-25b50a4c62b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:42.971309 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed589d-9b2f-4a38-9780-25b50a4c62b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:03:42.972134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed589d-9b2f-4a38-9780-25b50a4c62b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:42.973245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed589d-9b2f-4a38-9780-25b50a4c62b5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:42.979343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1086/2152] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:03:42.991376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1067/2153] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:03:42.992560 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225eb41d-40ee-46ca-9aec-d3ce1e75f023 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:43.003529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225eb41d-40ee-46ca-9aec-d3ce1e75f023 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:43.008501 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8563ab-fcbf-4cde-a43a-70b0fc592765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:43.009107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8563ab-fcbf-4cde-a43a-70b0fc592765 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:43.009387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8563ab-fcbf-4cde-a43a-70b0fc592765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:43.009656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8563ab-fcbf-4cde-a43a-70b0fc592765 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:43.010300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8563ab-fcbf-4cde-a43a-70b0fc592765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:43.012232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8563ab-fcbf-4cde-a43a-70b0fc592765 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:43.037239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:43.046631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1087/2154] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:42 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:03:43.051771 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed589d-9b2f-4a38-9780-25b50a4c62b5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:43.065974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed589d-9b2f-4a38-9780-25b50a4c62b5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:43.141867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b69a4-a6c2-4605-99d3-1333e2613f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:43.142536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b69a4-a6c2-4605-99d3-1333e2613f36 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:43.142830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b69a4-a6c2-4605-99d3-1333e2613f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:43.143136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b69a4-a6c2-4605-99d3-1333e2613f36 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:43.143972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b69a4-a6c2-4605-99d3-1333e2613f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:43.145794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b69a4-a6c2-4605-99d3-1333e2613f36 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:43.171868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1068/2155] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:43 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:43.176087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8563ab-fcbf-4cde-a43a-70b0fc592765 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:43.187910 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8563ab-fcbf-4cde-a43a-70b0fc592765 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:43.192749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e45a-7e35-4847-a32e-46b33a5e0ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:43.193683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e45a-7e35-4847-a32e-46b33a5e0ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:43.193983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e45a-7e35-4847-a32e-46b33a5e0ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:43.194227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e45a-7e35-4847-a32e-46b33a5e0ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:43.194984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80e45a-7e35-4847-a32e-46b33a5e0ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:43.196888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80e45a-7e35-4847-a32e-46b33a5e0ebc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:43.212010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1088/2156] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:43.215759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b69a4-a6c2-4605-99d3-1333e2613f36 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:43.227550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b69a4-a6c2-4605-99d3-1333e2613f36 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:43.232972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674435c-e72e-436e-9445-2ebc8f87bad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:43.233587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674435c-e72e-436e-9445-2ebc8f87bad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:43.233867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674435c-e72e-436e-9445-2ebc8f87bad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:43.234138 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674435c-e72e-436e-9445-2ebc8f87bad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:43.234780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674435c-e72e-436e-9445-2ebc8f87bad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:43.236570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674435c-e72e-436e-9445-2ebc8f87bad7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:43.251499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1069/2157] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:43.255718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e45a-7e35-4847-a32e-46b33a5e0ebc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:43.269686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e45a-7e35-4847-a32e-46b33a5e0ebc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:43.274970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c37f79-4bf6-4849-94ee-f526807e3390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:43.275795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c37f79-4bf6-4849-94ee-f526807e3390 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:43.276176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c37f79-4bf6-4849-94ee-f526807e3390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:43.281822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c37f79-4bf6-4849-94ee-f526807e3390 admin admin] PATH_INFO: `/v3/projects/73e4b51ff3bb4daca326d01859c2e62a/users/a299752caee04dc09bb26e4de59fa325/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:43.297199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c37f79-4bf6-4849-94ee-f526807e3390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e4b51ff3bb4daca326d01859c2e62a, user_id=a299752caee04dc09bb26e4de59fa325, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:43.300097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c37f79-4bf6-4849-94ee-f526807e3390 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:43.316600 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1089/2158] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:43 2022] PUT /identity/v3/projects/73e4b51ff3bb4daca326d01859c2e62a/users/a299752caee04dc09bb26e4de59fa325/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:43.329059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734efcd-946c-4de1-9eaa-9bb4086515a8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:43.329350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734efcd-946c-4de1-9eaa-9bb4086515a8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:43.329628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734efcd-946c-4de1-9eaa-9bb4086515a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:43.337000 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:43.362321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5734efcd-946c-4de1-9eaa-9bb4086515a8 None None] MFA Rules not processed for user `a299752caee04dc09bb26e4de59fa325`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:43.363624 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5734efcd-946c-4de1-9eaa-9bb4086515a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:43.438502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1070/2159] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:03:43.449960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c37f79-4bf6-4849-94ee-f526807e3390 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:43.452374 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-90c37f79-4bf6-4849-94ee-f526807e3390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:43.512431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c37f79-4bf6-4849-94ee-f526807e3390 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:43.516949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c562-8284-4329-8d8a-145eb65e1039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:43.517611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c562-8284-4329-8d8a-145eb65e1039 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:43.517887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c562-8284-4329-8d8a-145eb65e1039 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:43.518186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c562-8284-4329-8d8a-145eb65e1039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:43.527561 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74c562-8284-4329-8d8a-145eb65e1039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:43.529079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74c562-8284-4329-8d8a-145eb65e1039 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:43.538254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1090/2160] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:43.635269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734efcd-946c-4de1-9eaa-9bb4086515a8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:43.636684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5734efcd-946c-4de1-9eaa-9bb4086515a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:43.722468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734efcd-946c-4de1-9eaa-9bb4086515a8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:43.728127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:43.728893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:43.729305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:43.729588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] PATH_INFO: `/v3/users/10e43105015d477cb940eb3a6392090f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:43.731516 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e43105015d477cb940eb3a6392090f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:43.733409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:43.799582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e43105015d477cb940eb3a6392090f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:43.823159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e43105015d477cb940eb3a6392090f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:43.823159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e43105015d477cb940eb3a6392090f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:43.823159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e43105015d477cb940eb3a6392090f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:43.830908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e43105015d477cb940eb3a6392090f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:43.852117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1071/2161] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:43 2022] DELETE /identity/v3/users/10e43105015d477cb940eb3a6392090f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:43.853992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65a62c-a2fe-4ee7-935f-4d7460b6ee3e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:43.854275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65a62c-a2fe-4ee7-935f-4d7460b6ee3e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:43.854545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65a62c-a2fe-4ee7-935f-4d7460b6ee3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:43.894075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5a65a62c-a2fe-4ee7-935f-4d7460b6ee3e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:43.895416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5a65a62c-a2fe-4ee7-935f-4d7460b6ee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:43.966629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1091/2162] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:43.976899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:43.978502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:43.993380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:44.042761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d5078-1321-4241-ba57-b0cb57ed6f71 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:44.047306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c92f8-6f74-4218-89b9-16f26167708e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:44.047985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c92f8-6f74-4218-89b9-16f26167708e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:44.048320 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c92f8-6f74-4218-89b9-16f26167708e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:44.048606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c92f8-6f74-4218-89b9-16f26167708e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:44.058140 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c92f8-6f74-4218-89b9-16f26167708e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:44.059596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c92f8-6f74-4218-89b9-16f26167708e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:44.069871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1072/2163] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:44.347657 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65a62c-a2fe-4ee7-935f-4d7460b6ee3e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:44.349896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5a65a62c-a2fe-4ee7-935f-4d7460b6ee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:44.363670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65a62c-a2fe-4ee7-935f-4d7460b6ee3e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:44.368233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:44.368866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:44.369148 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:44.369422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] PATH_INFO: `/v3/projects/9543c8801b5b499682005c2f254208be` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:44.370532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9543c8801b5b499682005c2f254208be)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:44.372473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:44.447688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9543c8801b5b499682005c2f254208be', 'request_id': 'req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:44.456216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9543c8801b5b499682005c2f254208be', 'request_id': 'req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:44.463620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9543c8801b5b499682005c2f254208be', 'request_id': 'req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:44.464914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:44.478440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] The token cache is being invalidate because project 9543c8801b5b499682005c2f254208be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:44.478955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:44.479678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:44.481328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1092/2164] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:44 2022] DELETE /identity/v3/projects/9543c8801b5b499682005c2f254208be => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:44.485823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c92f8-6f74-4218-89b9-16f26167708e service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:44.497077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c92f8-6f74-4218-89b9-16f26167708e service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:44.554270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbda2c-d951-4cc7-a898-c6a57746d667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:44.554897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbda2c-d951-4cc7-a898-c6a57746d667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:44.555216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbda2c-d951-4cc7-a898-c6a57746d667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:44.555489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbda2c-d951-4cc7-a898-c6a57746d667 admin admin] PATH_INFO: `/v3/users/ac2240be1a4749a1a66f6544a0ba949e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:44.556603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbda2c-d951-4cc7-a898-c6a57746d667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2240be1a4749a1a66f6544a0ba949e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:44.558523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbda2c-d951-4cc7-a898-c6a57746d667 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:44.606006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ebbda2c-d951-4cc7-a898-c6a57746d667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2240be1a4749a1a66f6544a0ba949e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:44.617570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ebbda2c-d951-4cc7-a898-c6a57746d667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2240be1a4749a1a66f6544a0ba949e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:44.618230 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ebbda2c-d951-4cc7-a898-c6a57746d667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2240be1a4749a1a66f6544a0ba949e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:44.618825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ebbda2c-d951-4cc7-a898-c6a57746d667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2240be1a4749a1a66f6544a0ba949e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:44.626599 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ebbda2c-d951-4cc7-a898-c6a57746d667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2240be1a4749a1a66f6544a0ba949e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:44.640195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1073/2165] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:44 2022] DELETE /identity/v3/users/ac2240be1a4749a1a66f6544a0ba949e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:44.945337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:44.947257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:44.973094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2201a-e0ca-423a-8bfc-9db1afea07aa admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.023134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.023766 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.024043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.024348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] PATH_INFO: `/v3/projects/26c698be17d3418b8d5a93ee0e8251c3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.025666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26c698be17d3418b8d5a93ee0e8251c3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.027646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:45.073935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26c698be17d3418b8d5a93ee0e8251c3', 'request_id': 'req-b810c971-f329-4fce-9512-54e60a5c3a65'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:45.084649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26c698be17d3418b8d5a93ee0e8251c3', 'request_id': 'req-b810c971-f329-4fce-9512-54e60a5c3a65'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:45.091120 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26c698be17d3418b8d5a93ee0e8251c3', 'request_id': 'req-b810c971-f329-4fce-9512-54e60a5c3a65'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:45.102761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] The token cache is being invalidate because project 26c698be17d3418b8d5a93ee0e8251c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:45.103093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:45.103864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b810c971-f329-4fce-9512-54e60a5c3a65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:45.105432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1093/2166] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:44 2022] DELETE /identity/v3/projects/26c698be17d3418b8d5a93ee0e8251c3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:45.110435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39509d-479e-4230-9926-d3d561440f98 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.110727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39509d-479e-4230-9926-d3d561440f98 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.111035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39509d-479e-4230-9926-d3d561440f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.130945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6a39509d-479e-4230-9926-d3d561440f98 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:45.132690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6a39509d-479e-4230-9926-d3d561440f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:45.178914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:45.215268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1074/2167] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:03:45.223150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790061-d747-4518-9412-5e8103f77c35 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.223492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790061-d747-4518-9412-5e8103f77c35 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.223773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790061-d747-4518-9412-5e8103f77c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.244238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-40790061-d747-4518-9412-5e8103f77c35 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:45.245542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-40790061-d747-4518-9412-5e8103f77c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:45.254134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1094/2168] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:03:45.259037 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39509d-479e-4230-9926-d3d561440f98 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:45.269689 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39509d-479e-4230-9926-d3d561440f98 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.274481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588605e-25c9-47da-a29c-addc0c5b28f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.275053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588605e-25c9-47da-a29c-addc0c5b28f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.275339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588605e-25c9-47da-a29c-addc0c5b28f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.275614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588605e-25c9-47da-a29c-addc0c5b28f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.276266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588605e-25c9-47da-a29c-addc0c5b28f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:03:45.276572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588605e-25c9-47da-a29c-addc0c5b28f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.278164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588605e-25c9-47da-a29c-addc0c5b28f5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:45.291083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1075/2169] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:03:45.296055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40790061-d747-4518-9412-5e8103f77c35 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:45.305285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40790061-d747-4518-9412-5e8103f77c35 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.310947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b11949-ab51-4565-8c01-11d57b547c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.311569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11949-ab51-4565-8c01-11d57b547c37 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.311843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11949-ab51-4565-8c01-11d57b547c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.312108 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11949-ab51-4565-8c01-11d57b547c37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.312737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b11949-ab51-4565-8c01-11d57b547c37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.314753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b11949-ab51-4565-8c01-11d57b547c37 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:45.343868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1095/2170] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:45 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:45.348372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588605e-25c9-47da-a29c-addc0c5b28f5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:45.359427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588605e-25c9-47da-a29c-addc0c5b28f5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.418301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747bb85-bcc8-4b54-8ce9-120e480c32b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.418925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747bb85-bcc8-4b54-8ce9-120e480c32b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.419233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747bb85-bcc8-4b54-8ce9-120e480c32b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.419999 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747bb85-bcc8-4b54-8ce9-120e480c32b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.420703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747bb85-bcc8-4b54-8ce9-120e480c32b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.422694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747bb85-bcc8-4b54-8ce9-120e480c32b3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:45.444355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1076/2171] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:45 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:45.448624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b11949-ab51-4565-8c01-11d57b547c37 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:45.459940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b11949-ab51-4565-8c01-11d57b547c37 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.464743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd7213-c06a-4349-9ced-af387d345ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.465346 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd7213-c06a-4349-9ced-af387d345ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.465722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd7213-c06a-4349-9ced-af387d345ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.466065 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd7213-c06a-4349-9ced-af387d345ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.467597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd7213-c06a-4349-9ced-af387d345ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.469445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd7213-c06a-4349-9ced-af387d345ec4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:45.479620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1096/2172] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:45.483828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747bb85-bcc8-4b54-8ce9-120e480c32b3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:45.495018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747bb85-bcc8-4b54-8ce9-120e480c32b3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.499639 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc0d5f-86c9-4286-9c3d-4fd9151a5b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.500284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc0d5f-86c9-4286-9c3d-4fd9151a5b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.500569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc0d5f-86c9-4286-9c3d-4fd9151a5b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.500905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc0d5f-86c9-4286-9c3d-4fd9151a5b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.501557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc0d5f-86c9-4286-9c3d-4fd9151a5b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.503189 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc0d5f-86c9-4286-9c3d-4fd9151a5b35 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:45.523188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1077/2173] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:45.524360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd7213-c06a-4349-9ced-af387d345ec4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:45.534926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd7213-c06a-4349-9ced-af387d345ec4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.539419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb61a3a-6805-4582-b0bb-26de97f727db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.540064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb61a3a-6805-4582-b0bb-26de97f727db admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.540339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb61a3a-6805-4582-b0bb-26de97f727db admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.540611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb61a3a-6805-4582-b0bb-26de97f727db admin admin] PATH_INFO: `/v3/projects/c770ad8f3181429791ea705190d7d60a/users/8e0e41e8e48843089b2de504b0a35e4d/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.555675 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb61a3a-6805-4582-b0bb-26de97f727db admin admin] RBAC: Authorizing `identity:create_grant(project_id=c770ad8f3181429791ea705190d7d60a, user_id=8e0e41e8e48843089b2de504b0a35e4d, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.558329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb61a3a-6805-4582-b0bb-26de97f727db admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:45.571870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1097/2174] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:45 2022] PUT /identity/v3/projects/c770ad8f3181429791ea705190d7d60a/users/8e0e41e8e48843089b2de504b0a35e4d/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:45.583716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ac10-9217-4190-8942-17ef18076651 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.584011 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ac10-9217-4190-8942-17ef18076651 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.584469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ac10-9217-4190-8942-17ef18076651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.614027 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b396ac10-9217-4190-8942-17ef18076651 None None] MFA Rules not processed for user `8e0e41e8e48843089b2de504b0a35e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:45.615531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b396ac10-9217-4190-8942-17ef18076651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:45.682085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1078/2175] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:45.686756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb61a3a-6805-4582-b0bb-26de97f727db admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:45.705382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:45.751928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb61a3a-6805-4582-b0bb-26de97f727db admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.756554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343a632-017c-43c3-aa35-f7ad112c3b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.757168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343a632-017c-43c3-aa35-f7ad112c3b3e admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.757450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343a632-017c-43c3-aa35-f7ad112c3b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.757724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343a632-017c-43c3-aa35-f7ad112c3b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.758364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343a632-017c-43c3-aa35-f7ad112c3b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.760294 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343a632-017c-43c3-aa35-f7ad112c3b3e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:45.787196 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1098/2176] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:45 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:45.791185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396ac10-9217-4190-8942-17ef18076651 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:45.801962 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396ac10-9217-4190-8942-17ef18076651 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.829729 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:45.862875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd37d7-dd9a-49c6-82c5-3f3b67d76f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.863523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd37d7-dd9a-49c6-82c5-3f3b67d76f09 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.863847 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd37d7-dd9a-49c6-82c5-3f3b67d76f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.864121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd37d7-dd9a-49c6-82c5-3f3b67d76f09 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.864740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd37d7-dd9a-49c6-82c5-3f3b67d76f09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.866663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd37d7-dd9a-49c6-82c5-3f3b67d76f09 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:45.887942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1079/2177] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:45 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:45.892554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343a632-017c-43c3-aa35-f7ad112c3b3e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:45.903293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343a632-017c-43c3-aa35-f7ad112c3b3e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.908530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfb7f6-8d13-47e3-a06a-40b014e490ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.909133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb7f6-8d13-47e3-a06a-40b014e490ae admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.909416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb7f6-8d13-47e3-a06a-40b014e490ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.909691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb7f6-8d13-47e3-a06a-40b014e490ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.910328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfb7f6-8d13-47e3-a06a-40b014e490ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.911938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfb7f6-8d13-47e3-a06a-40b014e490ae admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:45.922407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1099/2178] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:03:45.926413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd37d7-dd9a-49c6-82c5-3f3b67d76f09 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:45.936542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd37d7-dd9a-49c6-82c5-3f3b67d76f09 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.941212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061831e-27a3-4296-aa2a-1a3c1e6c2c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.941804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061831e-27a3-4296-aa2a-1a3c1e6c2c48 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.942082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061831e-27a3-4296-aa2a-1a3c1e6c2c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.942356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061831e-27a3-4296-aa2a-1a3c1e6c2c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.943395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061831e-27a3-4296-aa2a-1a3c1e6c2c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.945067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061831e-27a3-4296-aa2a-1a3c1e6c2c48 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:45.956874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1080/2179] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:45.960905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfb7f6-8d13-47e3-a06a-40b014e490ae admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:45.971751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfb7f6-8d13-47e3-a06a-40b014e490ae admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:45.976666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c85898-abd8-4ecb-b5ca-3861d06e4a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:45.977588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c85898-abd8-4ecb-b5ca-3861d06e4a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:45.977733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c85898-abd8-4ecb-b5ca-3861d06e4a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:45.978012 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c85898-abd8-4ecb-b5ca-3861d06e4a50 admin admin] PATH_INFO: `/v3/projects/bdebcd1a9b60454c8a39509f6747f2dd/users/2c6d8791206447fcb67e624ceb0be675/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:45.993418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c85898-abd8-4ecb-b5ca-3861d06e4a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdebcd1a9b60454c8a39509f6747f2dd, user_id=2c6d8791206447fcb67e624ceb0be675, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:45.996104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c85898-abd8-4ecb-b5ca-3861d06e4a50 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:46.004627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061831e-27a3-4296-aa2a-1a3c1e6c2c48 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:46.007196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4061831e-27a3-4296-aa2a-1a3c1e6c2c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:46.015480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1100/2180] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:45 2022] PUT /identity/v3/projects/bdebcd1a9b60454c8a39509f6747f2dd/users/2c6d8791206447fcb67e624ceb0be675/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:46.024138 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061831e-27a3-4296-aa2a-1a3c1e6c2c48 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:46.025585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f23ac-fab5-4a0b-b395-fe46afcabfaa None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.026063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f23ac-fab5-4a0b-b395-fe46afcabfaa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.026551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f23ac-fab5-4a0b-b395-fe46afcabfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.058215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-967f23ac-fab5-4a0b-b395-fe46afcabfaa None None] MFA Rules not processed for user `2c6d8791206447fcb67e624ceb0be675`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:46.059495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-967f23ac-fab5-4a0b-b395-fe46afcabfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:46.085921 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b230938-6e8d-4cd9-a8da-ccde19de555b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:46.086557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b230938-6e8d-4cd9-a8da-ccde19de555b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.086840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b230938-6e8d-4cd9-a8da-ccde19de555b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.087248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b230938-6e8d-4cd9-a8da-ccde19de555b admin admin] PATH_INFO: `/v3/users/6baa720eb4504e44a5a4c5cb57d0ed73` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.090915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b230938-6e8d-4cd9-a8da-ccde19de555b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6baa720eb4504e44a5a4c5cb57d0ed73)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:46.092956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b230938-6e8d-4cd9-a8da-ccde19de555b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:46.145671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1101/2181] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:03:46.150074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f23ac-fab5-4a0b-b395-fe46afcabfaa None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:46.151067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3b230938-6e8d-4cd9-a8da-ccde19de555b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6baa720eb4504e44a5a4c5cb57d0ed73'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:46.160694 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f23ac-fab5-4a0b-b395-fe46afcabfaa None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:46.166376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a041b78-922c-4846-bf9e-76f76c300961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:46.167023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a041b78-922c-4846-bf9e-76f76c300961 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.167139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3b230938-6e8d-4cd9-a8da-ccde19de555b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6baa720eb4504e44a5a4c5cb57d0ed73'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:46.167311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a041b78-922c-4846-bf9e-76f76c300961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.167584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a041b78-922c-4846-bf9e-76f76c300961 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.167765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3b230938-6e8d-4cd9-a8da-ccde19de555b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6baa720eb4504e44a5a4c5cb57d0ed73'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:46.168272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a041b78-922c-4846-bf9e-76f76c300961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:46.168525 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3b230938-6e8d-4cd9-a8da-ccde19de555b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6baa720eb4504e44a5a4c5cb57d0ed73'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:46.170317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a041b78-922c-4846-bf9e-76f76c300961 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:46.178208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3b230938-6e8d-4cd9-a8da-ccde19de555b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6baa720eb4504e44a5a4c5cb57d0ed73'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:46.195152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1081/2182] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:46 2022] DELETE /identity/v3/users/6baa720eb4504e44a5a4c5cb57d0ed73 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:46.199770 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd751e3-f608-46b0-adb9-a0beaf669cc8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.201264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd751e3-f608-46b0-adb9-a0beaf669cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.201867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd751e3-f608-46b0-adb9-a0beaf669cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.207028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1102/2183] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:46 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:46.208144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a041b78-922c-4846-bf9e-76f76c300961 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:46.209811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3a041b78-922c-4846-bf9e-76f76c300961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:46.223466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0cd751e3-f608-46b0-adb9-a0beaf669cc8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:46.225190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0cd751e3-f608-46b0-adb9-a0beaf669cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:46.251783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:46.255107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:46.306197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a041b78-922c-4846-bf9e-76f76c300961 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:46.312503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942078d6-ccfc-4535-a139-2d6fb3a129be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:46.313064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1082/2184] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:03:46.313338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942078d6-ccfc-4535-a139-2d6fb3a129be admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.313774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942078d6-ccfc-4535-a139-2d6fb3a129be admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.314154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942078d6-ccfc-4535-a139-2d6fb3a129be admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.315266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942078d6-ccfc-4535-a139-2d6fb3a129be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:46.317524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942078d6-ccfc-4535-a139-2d6fb3a129be admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:46.324796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd751e3-f608-46b0-adb9-a0beaf669cc8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:46.328243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0cd751e3-f608-46b0-adb9-a0beaf669cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:46.350130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1103/2185] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:46 2022] POST /identity/v3/users => generated 298 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:03:46.351491 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942078d6-ccfc-4535-a139-2d6fb3a129be admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:46.366895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942078d6-ccfc-4535-a139-2d6fb3a129be admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:46.400091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b30de-dd0f-4b7d-9fe9-75b3b971d5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:46.400741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b30de-dd0f-4b7d-9fe9-75b3b971d5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.401022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b30de-dd0f-4b7d-9fe9-75b3b971d5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.401293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b30de-dd0f-4b7d-9fe9-75b3b971d5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.402107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b30de-dd0f-4b7d-9fe9-75b3b971d5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:46.404722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b30de-dd0f-4b7d-9fe9-75b3b971d5a4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:46.425848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd751e3-f608-46b0-adb9-a0beaf669cc8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:46.437353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49b1a9-2baf-41ed-b8a9-1f510b57037b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:46.439278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1104/2186] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:46.440114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49b1a9-2baf-41ed-b8a9-1f510b57037b service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.440228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b30de-dd0f-4b7d-9fe9-75b3b971d5a4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:46.440678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49b1a9-2baf-41ed-b8a9-1f510b57037b service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.441197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49b1a9-2baf-41ed-b8a9-1f510b57037b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.456368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b30de-dd0f-4b7d-9fe9-75b3b971d5a4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:46.465691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35427c7c-291f-41de-925c-5013acc4b459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:46.466562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35427c7c-291f-41de-925c-5013acc4b459 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.466847 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35427c7c-291f-41de-925c-5013acc4b459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.467708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35427c7c-291f-41de-925c-5013acc4b459 admin admin] PATH_INFO: `/v3/projects/947e8640877d49ea91558fb30b67ceda/users/bca220f4588b43b8a82a8e50fd2187a7/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.483162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49b1a9-2baf-41ed-b8a9-1f510b57037b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:46.486413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49b1a9-2baf-41ed-b8a9-1f510b57037b service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:46.486565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35427c7c-291f-41de-925c-5013acc4b459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=947e8640877d49ea91558fb30b67ceda, user_id=bca220f4588b43b8a82a8e50fd2187a7, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:46.490517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35427c7c-291f-41de-925c-5013acc4b459 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:46.516378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1105/2187] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:46 2022] PUT /identity/v3/projects/947e8640877d49ea91558fb30b67ceda/users/bca220f4588b43b8a82a8e50fd2187a7/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:46.520715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35427c7c-291f-41de-925c-5013acc4b459 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:46.526923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1083/2188] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:46.623948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35427c7c-291f-41de-925c-5013acc4b459 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:46.631393 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851027e-a7de-4799-9c6d-ef232b221561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:46.632051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851027e-a7de-4799-9c6d-ef232b221561 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.632331 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851027e-a7de-4799-9c6d-ef232b221561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.632629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851027e-a7de-4799-9c6d-ef232b221561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.633293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851027e-a7de-4799-9c6d-ef232b221561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:46.635037 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851027e-a7de-4799-9c6d-ef232b221561 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:46.647258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1106/2189] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:46.651425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49b1a9-2baf-41ed-b8a9-1f510b57037b service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:46.662803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49b1a9-2baf-41ed-b8a9-1f510b57037b service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:46.667692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d0d28-cba6-4edb-86f3-3058eba9d7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:46.668485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d0d28-cba6-4edb-86f3-3058eba9d7cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.668778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d0d28-cba6-4edb-86f3-3058eba9d7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.669127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d0d28-cba6-4edb-86f3-3058eba9d7cc admin admin] PATH_INFO: `/v3/projects/947e8640877d49ea91558fb30b67ceda/users/bca220f4588b43b8a82a8e50fd2187a7/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.673261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d0d28-cba6-4edb-86f3-3058eba9d7cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=947e8640877d49ea91558fb30b67ceda, user_id=bca220f4588b43b8a82a8e50fd2187a7, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:46.679523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d0d28-cba6-4edb-86f3-3058eba9d7cc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:46.701603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1084/2190] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:46 2022] PUT /identity/v3/projects/947e8640877d49ea91558fb30b67ceda/users/bca220f4588b43b8a82a8e50fd2187a7/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:46.711345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1f3c3-432b-4795-a245-118ea0df58d1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.711703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1f3c3-432b-4795-a245-118ea0df58d1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.712081 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1f3c3-432b-4795-a245-118ea0df58d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.749370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0cb1f3c3-432b-4795-a245-118ea0df58d1 None None] MFA Rules not processed for user `bca220f4588b43b8a82a8e50fd2187a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:46.750689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0cb1f3c3-432b-4795-a245-118ea0df58d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:46.832650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1107/2191] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:03:46.845247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d0d28-cba6-4edb-86f3-3058eba9d7cc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:46.847433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-337d0d28-cba6-4edb-86f3-3058eba9d7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:46.927639 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d0d28-cba6-4edb-86f3-3058eba9d7cc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:46.931279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51a40-ffc6-4b02-a9e8-e012ff6e3fac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:46.931888 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51a40-ffc6-4b02-a9e8-e012ff6e3fac service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:46.932164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51a40-ffc6-4b02-a9e8-e012ff6e3fac service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:46.932552 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51a40-ffc6-4b02-a9e8-e012ff6e3fac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:46.934276 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-95c51a40-ffc6-4b02-a9e8-e012ff6e3fac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:46.993133 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c51a40-ffc6-4b02-a9e8-e012ff6e3fac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:46.994933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c51a40-ffc6-4b02-a9e8-e012ff6e3fac service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:47.003591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1085/2192] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:47.074006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1f3c3-432b-4795-a245-118ea0df58d1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:47.075665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0cb1f3c3-432b-4795-a245-118ea0df58d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:47.088314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:47.169460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1f3c3-432b-4795-a245-118ea0df58d1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:47.175066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:47.175313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:47.175596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:47.175870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] PATH_INFO: `/v3/projects/91ea292631c3433699fa30b052e73b4f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:47.177635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ea292631c3433699fa30b052e73b4f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:47.179362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:47.245213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ea292631c3433699fa30b052e73b4f', 'request_id': 'req-eb24b2a9-0942-4e61-90eb-6d7f288ee688'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.255530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ea292631c3433699fa30b052e73b4f', 'request_id': 'req-eb24b2a9-0942-4e61-90eb-6d7f288ee688'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.265358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ea292631c3433699fa30b052e73b4f', 'request_id': 'req-eb24b2a9-0942-4e61-90eb-6d7f288ee688'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.281704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] The token cache is being invalidate because project 91ea292631c3433699fa30b052e73b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:47.282142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.283257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.286053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1108/2193] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:47 2022] DELETE /identity/v3/projects/91ea292631c3433699fa30b052e73b4f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:47.290268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51a40-ffc6-4b02-a9e8-e012ff6e3fac service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:47.308469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51a40-ffc6-4b02-a9e8-e012ff6e3fac service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:47.317381 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:47.380201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:47.380614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:47.380892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:47.381161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] PATH_INFO: `/v3/users/c2041104ac3e4394b79cd7045b0c6633` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:47.382290 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2041104ac3e4394b79cd7045b0c6633)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:47.384222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:47.436748 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2041104ac3e4394b79cd7045b0c6633'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.453091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2041104ac3e4394b79cd7045b0c6633'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.453864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2041104ac3e4394b79cd7045b0c6633'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.454441 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2041104ac3e4394b79cd7045b0c6633'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.462070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2041104ac3e4394b79cd7045b0c6633'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.477451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1086/2194] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:47 2022] DELETE /identity/v3/users/c2041104ac3e4394b79cd7045b0c6633 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:47.780409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:47.781887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:47.796029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24b2a9-0942-4e61-90eb-6d7f288ee688 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:47.859017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:47.859656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:47.859934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:47.860200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] PATH_INFO: `/v3/projects/96a0839fc1664049bc85133be46165ed` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:47.861444 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a0839fc1664049bc85133be46165ed)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:47.863386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:47.907783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:47.917052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a0839fc1664049bc85133be46165ed', 'request_id': 'req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.925426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a0839fc1664049bc85133be46165ed', 'request_id': 'req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.931923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a0839fc1664049bc85133be46165ed', 'request_id': 'req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.944268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] The token cache is being invalidate because project 96a0839fc1664049bc85133be46165ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:47.944556 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.945221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:47.946804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1109/2195] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:47 2022] DELETE /identity/v3/projects/96a0839fc1664049bc85133be46165ed => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:48.619816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:48.621282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:48.692377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9664dc0-ce0f-44a7-be26-35ad984f4800 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:48.743019 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:48.749945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9615b-b0b6-4472-b96b-67749c51604b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:48.750882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9615b-b0b6-4472-b96b-67749c51604b service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:48.752899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9615b-b0b6-4472-b96b-67749c51604b service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:48.753206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9615b-b0b6-4472-b96b-67749c51604b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:48.755421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-90b9615b-b0b6-4472-b96b-67749c51604b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:48.826276 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9615b-b0b6-4472-b96b-67749c51604b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:48.828183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9615b-b0b6-4472-b96b-67749c51604b service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:48.842318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1087/2196] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:49.500832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:49.502232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:49.515447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d6559-5e33-4f9c-90a7-6c82a5f0ecf7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:49.518974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7166e-0fc9-43f3-b4af-761787a217d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:49.519597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7166e-0fc9-43f3-b4af-761787a217d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:49.519877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7166e-0fc9-43f3-b4af-761787a217d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:49.520150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7166e-0fc9-43f3-b4af-761787a217d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:49.521789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b7b7166e-0fc9-43f3-b4af-761787a217d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:49.583750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7166e-0fc9-43f3-b4af-761787a217d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:49.585242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7166e-0fc9-43f3-b4af-761787a217d7 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:49.595779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1110/2197] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:54.347304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9615b-b0b6-4472-b96b-67749c51604b service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:54.348871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-90b9615b-b0b6-4472-b96b-67749c51604b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:54.428402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9615b-b0b6-4472-b96b-67749c51604b service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:54.432958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:54.433593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:54.433883 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:54.434184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] PATH_INFO: `/v3/users/a299752caee04dc09bb26e4de59fa325` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:54.436066 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a299752caee04dc09bb26e4de59fa325)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:54.438009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:54.488719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a299752caee04dc09bb26e4de59fa325'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:54.501383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a299752caee04dc09bb26e4de59fa325'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:54.502060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a299752caee04dc09bb26e4de59fa325'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:54.502667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a299752caee04dc09bb26e4de59fa325'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:54.511025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a299752caee04dc09bb26e4de59fa325'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:54.523086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1088/2198] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:54 2022] DELETE /identity/v3/users/a299752caee04dc09bb26e4de59fa325 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:54.527501 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b2e2d-f1b2-45d8-80f6-830a6549907d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:54.527957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b2e2d-f1b2-45d8-80f6-830a6549907d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:54.528070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b2e2d-f1b2-45d8-80f6-830a6549907d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:54.547153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-505b2e2d-f1b2-45d8-80f6-830a6549907d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:54.548337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-505b2e2d-f1b2-45d8-80f6-830a6549907d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:54.609956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:54.610609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1111/2199] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:54.611721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:54.620110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b2e2d-f1b2-45d8-80f6-830a6549907d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:54.621599 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-505b2e2d-f1b2-45d8-80f6-830a6549907d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:54.626251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0f528-6c5c-4f2e-aacf-3d06e4ae23a7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:54.631339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:54.631969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:54.632245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:54.632515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] PATH_INFO: `/v3/users/8e0e41e8e48843089b2de504b0a35e4d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:54.633655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e0e41e8e48843089b2de504b0a35e4d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:54.635601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:54.650763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:54.685238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b2e2d-f1b2-45d8-80f6-830a6549907d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:54.689236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809d9c9-7115-4ee4-af83-046c250c8f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:54.689853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809d9c9-7115-4ee4-af83-046c250c8f09 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:54.690145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809d9c9-7115-4ee4-af83-046c250c8f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:54.691100 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809d9c9-7115-4ee4-af83-046c250c8f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:54.691723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e0e41e8e48843089b2de504b0a35e4d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:54.701254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809d9c9-7115-4ee4-af83-046c250c8f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:54.702763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809d9c9-7115-4ee4-af83-046c250c8f09 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:54.705491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e0e41e8e48843089b2de504b0a35e4d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:54.706241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e0e41e8e48843089b2de504b0a35e4d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:54.706982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e0e41e8e48843089b2de504b0a35e4d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:54.712652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1112/2200] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:03:54.716856 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e0e41e8e48843089b2de504b0a35e4d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:54.729480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1089/2201] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:54 2022] DELETE /identity/v3/users/8e0e41e8e48843089b2de504b0a35e4d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:54.734545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6066-11ab-4909-8c75-196e2a6de831 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:54.734894 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6066-11ab-4909-8c75-196e2a6de831 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:54.735286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6066-11ab-4909-8c75-196e2a6de831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:54.748612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:54.756017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bcdb6066-11ab-4909-8c75-196e2a6de831 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:54.758292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bcdb6066-11ab-4909-8c75-196e2a6de831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:54.823794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1113/2202] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:54.833719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:54.835369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:54.906826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871249ec-43ac-420c-84d8-82605e9a367e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:54.911120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14d3e8-4a3a-4723-b9ae-16cbd304ed2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:54.911790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d3e8-4a3a-4723-b9ae-16cbd304ed2e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:54.912072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d3e8-4a3a-4723-b9ae-16cbd304ed2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:54.912348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d3e8-4a3a-4723-b9ae-16cbd304ed2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:54.922560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14d3e8-4a3a-4723-b9ae-16cbd304ed2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:54.924058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14d3e8-4a3a-4723-b9ae-16cbd304ed2e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:54.933650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1090/2203] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:55.034828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb6066-11ab-4909-8c75-196e2a6de831 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:55.036337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bcdb6066-11ab-4909-8c75-196e2a6de831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:55.054508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb6066-11ab-4909-8c75-196e2a6de831 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:55.060194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:55.060838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.061114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.061385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] PATH_INFO: `/v3/projects/73e4b51ff3bb4daca326d01859c2e62a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.062523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e4b51ff3bb4daca326d01859c2e62a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:55.064916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:55.114850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e4b51ff3bb4daca326d01859c2e62a', 'request_id': 'req-f984ac67-885b-4dca-abe2-24d73e82e7de'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.122572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e4b51ff3bb4daca326d01859c2e62a', 'request_id': 'req-f984ac67-885b-4dca-abe2-24d73e82e7de'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.129142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e4b51ff3bb4daca326d01859c2e62a', 'request_id': 'req-f984ac67-885b-4dca-abe2-24d73e82e7de'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.147316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] The token cache is being invalidate because project 73e4b51ff3bb4daca326d01859c2e62a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:55.147628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.148273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f984ac67-885b-4dca-abe2-24d73e82e7de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.150107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1114/2204] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:55 2022] DELETE /identity/v3/projects/73e4b51ff3bb4daca326d01859c2e62a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:55.155377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e03042-69dd-4258-b85d-e68ef602da28 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.155668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e03042-69dd-4258-b85d-e68ef602da28 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.155942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e03042-69dd-4258-b85d-e68ef602da28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.176393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-59e03042-69dd-4258-b85d-e68ef602da28 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:55.177673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-59e03042-69dd-4258-b85d-e68ef602da28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:55.239038 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1091/2205] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:55.248545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89baf-2052-4ae9-8f3f-f062bf56b7b5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.248836 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89baf-2052-4ae9-8f3f-f062bf56b7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.249112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89baf-2052-4ae9-8f3f-f062bf56b7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.272576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e03042-69dd-4258-b85d-e68ef602da28 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:55.274008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-59e03042-69dd-4258-b85d-e68ef602da28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:55.281679 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2ce89baf-2052-4ae9-8f3f-f062bf56b7b5 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:55.282956 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2ce89baf-2052-4ae9-8f3f-f062bf56b7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:55.290130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e03042-69dd-4258-b85d-e68ef602da28 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:55.292905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1115/2206] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:55.296827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce89baf-2052-4ae9-8f3f-f062bf56b7b5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:55.301987 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:55.302650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.302926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.303258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] PATH_INFO: `/v3/projects/c770ad8f3181429791ea705190d7d60a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.304729 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c770ad8f3181429791ea705190d7d60a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:55.306671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:55.307970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce89baf-2052-4ae9-8f3f-f062bf56b7b5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:55.313623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d5d47-69e8-4b33-bcbc-333fd53de4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:55.314126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5d47-69e8-4b33-bcbc-333fd53de4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.314413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5d47-69e8-4b33-bcbc-333fd53de4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.314688 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5d47-69e8-4b33-bcbc-333fd53de4f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.315368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d5d47-69e8-4b33-bcbc-333fd53de4f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:03:55.316152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d5d47-69e8-4b33-bcbc-333fd53de4f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:55.318409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d5d47-69e8-4b33-bcbc-333fd53de4f0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:55.331320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1116/2207] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:03:55.335788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d5d47-69e8-4b33-bcbc-333fd53de4f0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:55.345241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:55.346766 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d5d47-69e8-4b33-bcbc-333fd53de4f0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:55.359859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c770ad8f3181429791ea705190d7d60a', 'request_id': 'req-c7561205-592d-4172-80d6-8e4a6b9fc3cc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.366293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c770ad8f3181429791ea705190d7d60a', 'request_id': 'req-c7561205-592d-4172-80d6-8e4a6b9fc3cc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.373246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c770ad8f3181429791ea705190d7d60a', 'request_id': 'req-c7561205-592d-4172-80d6-8e4a6b9fc3cc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.389560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] The token cache is being invalidate because project c770ad8f3181429791ea705190d7d60a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:55.389930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.390592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.392292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1092/2208] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:55 2022] DELETE /identity/v3/projects/c770ad8f3181429791ea705190d7d60a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:55.396075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:55.397645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:55.430642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f4e4-ff43-42d5-960a-809f0c80c295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:55.431664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f4e4-ff43-42d5-960a-809f0c80c295 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.432139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f4e4-ff43-42d5-960a-809f0c80c295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.433358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f4e4-ff43-42d5-960a-809f0c80c295 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.434444 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250f4e4-ff43-42d5-960a-809f0c80c295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:55.439190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250f4e4-ff43-42d5-960a-809f0c80c295 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:55.462523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:55.485114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1117/2209] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:55 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:55.486573 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f4e4-ff43-42d5-960a-809f0c80c295 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:55.487432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7561205-592d-4172-80d6-8e4a6b9fc3cc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:55.488880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8250f4e4-ff43-42d5-960a-809f0c80c295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:55.595480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:55.596731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.596979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.597447 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] PATH_INFO: `/v3/users/2c6d8791206447fcb67e624ceb0be675` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.599632 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6d8791206447fcb67e624ceb0be675)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:55.603240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:55.616771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f4e4-ff43-42d5-960a-809f0c80c295 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:55.626357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903b77f-be85-449e-ab47-1acd08af78cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:55.627157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b77f-be85-449e-ab47-1acd08af78cd admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.627438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b77f-be85-449e-ab47-1acd08af78cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.627712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b77f-be85-449e-ab47-1acd08af78cd admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.629606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903b77f-be85-449e-ab47-1acd08af78cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:55.631900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903b77f-be85-449e-ab47-1acd08af78cd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:55.665251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1118/2210] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:55 2022] POST /identity/v3/users => generated 282 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:03:55.666814 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903b77f-be85-449e-ab47-1acd08af78cd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:55.678967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903b77f-be85-449e-ab47-1acd08af78cd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:55.682726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6d8791206447fcb67e624ceb0be675'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.695267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c878c-659a-4c38-bf5e-b94e4936cfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:55.696811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c878c-659a-4c38-bf5e-b94e4936cfd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.697292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c878c-659a-4c38-bf5e-b94e4936cfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.697752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c878c-659a-4c38-bf5e-b94e4936cfd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.698846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c878c-659a-4c38-bf5e-b94e4936cfd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:55.708058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c878c-659a-4c38-bf5e-b94e4936cfd1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:55.717673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6d8791206447fcb67e624ceb0be675'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.718859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6d8791206447fcb67e624ceb0be675'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.720394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6d8791206447fcb67e624ceb0be675'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.725897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1119/2211] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:55.727624 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:55.731420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c878c-659a-4c38-bf5e-b94e4936cfd1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:55.735767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6d8791206447fcb67e624ceb0be675'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:55.736521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b74c878c-659a-4c38-bf5e-b94e4936cfd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:55.755335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c878c-659a-4c38-bf5e-b94e4936cfd1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:55.756115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1093/2212] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:55 2022] DELETE /identity/v3/users/2c6d8791206447fcb67e624ceb0be675 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:55.829058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9932433-9f71-4399-af19-a9b04689a53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:55.829677 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9932433-9f71-4399-af19-a9b04689a53d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.829964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9932433-9f71-4399-af19-a9b04689a53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.830238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9932433-9f71-4399-af19-a9b04689a53d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.830899 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9932433-9f71-4399-af19-a9b04689a53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:55.832522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9932433-9f71-4399-af19-a9b04689a53d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:55.843857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1120/2213] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:55.848395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:55.859142 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963b6e2-d8d7-485d-9a67-910fdfcfa540 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:55.864444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a0b9b-3b8f-4550-bff8-e798883a526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:55.865104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a0b9b-3b8f-4550-bff8-e798883a526f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.865399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a0b9b-3b8f-4550-bff8-e798883a526f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.865675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a0b9b-3b8f-4550-bff8-e798883a526f admin admin] PATH_INFO: `/v3/projects/c7ae59ef02f640a5988fc8b1debfbc59/users/516ee32b9f6a46d1a4e3690fc036ebe8/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.881366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a0b9b-3b8f-4550-bff8-e798883a526f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ae59ef02f640a5988fc8b1debfbc59, user_id=516ee32b9f6a46d1a4e3690fc036ebe8, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:55.884135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a0b9b-3b8f-4550-bff8-e798883a526f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:55.906024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1094/2214] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:55 2022] PUT /identity/v3/projects/c7ae59ef02f640a5988fc8b1debfbc59/users/516ee32b9f6a46d1a4e3690fc036ebe8/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:55.917031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34436527-3540-4fa0-978b-8b26f46cd5bd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.917665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34436527-3540-4fa0-978b-8b26f46cd5bd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.918007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34436527-3540-4fa0-978b-8b26f46cd5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.950021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-34436527-3540-4fa0-978b-8b26f46cd5bd None None] MFA Rules not processed for user `516ee32b9f6a46d1a4e3690fc036ebe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:55.952054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-34436527-3540-4fa0-978b-8b26f46cd5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.032417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:56.039107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1121/2215] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:56.041834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a0b9b-3b8f-4550-bff8-e798883a526f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.085633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34436527-3540-4fa0-978b-8b26f46cd5bd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.087578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-34436527-3540-4fa0-978b-8b26f46cd5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.118597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a0b9b-3b8f-4550-bff8-e798883a526f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.125908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87f98a-dc4c-4f4f-ad7f-df4154d73507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.126635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87f98a-dc4c-4f4f-ad7f-df4154d73507 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.126894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87f98a-dc4c-4f4f-ad7f-df4154d73507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.127190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87f98a-dc4c-4f4f-ad7f-df4154d73507 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.132310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87f98a-dc4c-4f4f-ad7f-df4154d73507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.139718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87f98a-dc4c-4f4f-ad7f-df4154d73507 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.175455 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1095/2216] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:56 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:03:56.180089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87f98a-dc4c-4f4f-ad7f-df4154d73507 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.195863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34436527-3540-4fa0-978b-8b26f46cd5bd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.259953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87f98a-dc4c-4f4f-ad7f-df4154d73507 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.267165 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cba82-9f5f-4c5b-b1ac-df7c7b5fa11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.267920 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cba82-9f5f-4c5b-b1ac-df7c7b5fa11f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.268199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cba82-9f5f-4c5b-b1ac-df7c7b5fa11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.268477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cba82-9f5f-4c5b-b1ac-df7c7b5fa11f admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.269118 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cba82-9f5f-4c5b-b1ac-df7c7b5fa11f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.271058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cba82-9f5f-4c5b-b1ac-df7c7b5fa11f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.273312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.274139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.274482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.274845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] PATH_INFO: `/v3/projects/bdebcd1a9b60454c8a39509f6747f2dd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.276246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdebcd1a9b60454c8a39509f6747f2dd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.278567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.298056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1096/2217] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:56 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:03:56.302087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cba82-9f5f-4c5b-b1ac-df7c7b5fa11f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.314621 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cba82-9f5f-4c5b-b1ac-df7c7b5fa11f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.320444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85c6e5-3fc4-451f-94d9-ebad4c9b997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.321617 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85c6e5-3fc4-451f-94d9-ebad4c9b997d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.322049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85c6e5-3fc4-451f-94d9-ebad4c9b997d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.322475 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85c6e5-3fc4-451f-94d9-ebad4c9b997d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.323168 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85c6e5-3fc4-451f-94d9-ebad4c9b997d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.324936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85c6e5-3fc4-451f-94d9-ebad4c9b997d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.338331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1097/2218] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:56.343016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdebcd1a9b60454c8a39509f6747f2dd', 'request_id': 'req-18b19fc5-e810-4c0e-8efb-91f42030e5b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:56.343768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85c6e5-3fc4-451f-94d9-ebad4c9b997d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.353417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdebcd1a9b60454c8a39509f6747f2dd', 'request_id': 'req-18b19fc5-e810-4c0e-8efb-91f42030e5b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:56.356793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85c6e5-3fc4-451f-94d9-ebad4c9b997d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.360364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdebcd1a9b60454c8a39509f6747f2dd', 'request_id': 'req-18b19fc5-e810-4c0e-8efb-91f42030e5b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:56.372336 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] The token cache is being invalidate because project bdebcd1a9b60454c8a39509f6747f2dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:56.373274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:56.373959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:56.375508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1122/2219] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:56 2022] DELETE /identity/v3/projects/bdebcd1a9b60454c8a39509f6747f2dd => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:56.379213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.382142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.384861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:56.437225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:56.444725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225d863-4819-42ae-a5e6-6da016094ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.445402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225d863-4819-42ae-a5e6-6da016094ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.445688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225d863-4819-42ae-a5e6-6da016094ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.445978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225d863-4819-42ae-a5e6-6da016094ba5 admin admin] PATH_INFO: `/v3/projects/395327af56004c41b6d27babf808a320/users/8bb5bcd3916b4cfa810169687a30d267/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.463808 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225d863-4819-42ae-a5e6-6da016094ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=395327af56004c41b6d27babf808a320, user_id=8bb5bcd3916b4cfa810169687a30d267, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.466718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225d863-4819-42ae-a5e6-6da016094ba5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.466835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b19fc5-e810-4c0e-8efb-91f42030e5b5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.472207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.472836 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.473115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.473390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] PATH_INFO: `/v3/users/bca220f4588b43b8a82a8e50fd2187a7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.474597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca220f4588b43b8a82a8e50fd2187a7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.476665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.483334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1098/2220] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:56 2022] PUT /identity/v3/projects/395327af56004c41b6d27babf808a320/users/8bb5bcd3916b4cfa810169687a30d267/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:56.487245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225d863-4819-42ae-a5e6-6da016094ba5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.488924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9225d863-4819-42ae-a5e6-6da016094ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.538019 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca220f4588b43b8a82a8e50fd2187a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:56.539603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca220f4588b43b8a82a8e50fd2187a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:56.551580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca220f4588b43b8a82a8e50fd2187a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:56.561332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca220f4588b43b8a82a8e50fd2187a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:56.566285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225d863-4819-42ae-a5e6-6da016094ba5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.571251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca220f4588b43b8a82a8e50fd2187a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:56.572208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1430c85-b116-42ef-9c80-64a8d0721a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.572896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430c85-b116-42ef-9c80-64a8d0721a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.573177 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430c85-b116-42ef-9c80-64a8d0721a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.573485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430c85-b116-42ef-9c80-64a8d0721a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.574122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1430c85-b116-42ef-9c80-64a8d0721a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.575770 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1430c85-b116-42ef-9c80-64a8d0721a9c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.579023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1123/2221] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:56 2022] DELETE /identity/v3/users/bca220f4588b43b8a82a8e50fd2187a7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:56.587409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1099/2222] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:03:56.591671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.594487 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.710363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a905371-0b49-40ee-b0a5-d926902f862c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.717139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177aa39-cfa2-4536-b674-1f192c4c672e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.717550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177aa39-cfa2-4536-b674-1f192c4c672e admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.717770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177aa39-cfa2-4536-b674-1f192c4c672e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.717959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177aa39-cfa2-4536-b674-1f192c4c672e admin admin] PATH_INFO: `/v3/projects/395327af56004c41b6d27babf808a320/users/8bb5bcd3916b4cfa810169687a30d267/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.720061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177aa39-cfa2-4536-b674-1f192c4c672e admin admin] RBAC: Authorizing `identity:create_grant(project_id=395327af56004c41b6d27babf808a320, user_id=8bb5bcd3916b4cfa810169687a30d267, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.722758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177aa39-cfa2-4536-b674-1f192c4c672e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.734512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1124/2223] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:56 2022] PUT /identity/v3/projects/395327af56004c41b6d27babf808a320/users/8bb5bcd3916b4cfa810169687a30d267/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:56.745682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8462c-6cdb-4a7d-8b49-a729419e938e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.745976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8462c-6cdb-4a7d-8b49-a729419e938e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.746272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8462c-6cdb-4a7d-8b49-a729419e938e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.793765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c4c8462c-6cdb-4a7d-8b49-a729419e938e None None] MFA Rules not processed for user `8bb5bcd3916b4cfa810169687a30d267`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:56.795088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c4c8462c-6cdb-4a7d-8b49-a729419e938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.875997 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1100/2224] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:56.883901 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177aa39-cfa2-4536-b674-1f192c4c672e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.886498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6177aa39-cfa2-4536-b674-1f192c4c672e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.967057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177aa39-cfa2-4536-b674-1f192c4c672e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.974106 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e3ade-03ef-494a-82cb-3726306ed333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.974736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e3ade-03ef-494a-82cb-3726306ed333 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.975164 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e3ade-03ef-494a-82cb-3726306ed333 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.975671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e3ade-03ef-494a-82cb-3726306ed333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.987919 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:56.990967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e3ade-03ef-494a-82cb-3726306ed333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.004928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e3ade-03ef-494a-82cb-3726306ed333 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.007942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1125/2225] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:57.057749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8462c-6cdb-4a7d-8b49-a729419e938e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.059475 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c4c8462c-6cdb-4a7d-8b49-a729419e938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.117195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:57.169184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8462c-6cdb-4a7d-8b49-a729419e938e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:57.175065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:57.175766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.176106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.176477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] PATH_INFO: `/v3/projects/947e8640877d49ea91558fb30b67ceda` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.177954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=947e8640877d49ea91558fb30b67ceda)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.180284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.235898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '947e8640877d49ea91558fb30b67ceda', 'request_id': 'req-9c77c207-66b0-4c8e-8130-863932e98659'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:57.246881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '947e8640877d49ea91558fb30b67ceda', 'request_id': 'req-9c77c207-66b0-4c8e-8130-863932e98659'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:57.248395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '947e8640877d49ea91558fb30b67ceda', 'request_id': 'req-9c77c207-66b0-4c8e-8130-863932e98659'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:57.261960 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] The token cache is being invalidate because project 947e8640877d49ea91558fb30b67ceda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:57.262637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:57.263984 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9c77c207-66b0-4c8e-8130-863932e98659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:57.265978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1101/2226] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:57 2022] DELETE /identity/v3/projects/947e8640877d49ea91558fb30b67ceda => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:57.271010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63a9c-0e52-4228-a706-9b2977be72eb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.271354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63a9c-0e52-4228-a706-9b2977be72eb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.271628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63a9c-0e52-4228-a706-9b2977be72eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.292088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-59c63a9c-0e52-4228-a706-9b2977be72eb None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:57.293345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-59c63a9c-0e52-4228-a706-9b2977be72eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.358387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1126/2227] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:57.366470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb05ae-1561-410a-bfed-8db2ec4c6d08 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.366958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb05ae-1561-410a-bfed-8db2ec4c6d08 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.367288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb05ae-1561-410a-bfed-8db2ec4c6d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.387670 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0bdb05ae-1561-410a-bfed-8db2ec4c6d08 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:57.389401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0bdb05ae-1561-410a-bfed-8db2ec4c6d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.397825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1102/2228] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:57.401748 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c63a9c-0e52-4228-a706-9b2977be72eb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.412194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c63a9c-0e52-4228-a706-9b2977be72eb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:57.418253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f79332-b769-4582-9ea4-f57002824093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:57.419238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f79332-b769-4582-9ea4-f57002824093 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.419697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f79332-b769-4582-9ea4-f57002824093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.419867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f79332-b769-4582-9ea4-f57002824093 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.420451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f79332-b769-4582-9ea4-f57002824093 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:03:57.423250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f79332-b769-4582-9ea4-f57002824093 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.425155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f79332-b769-4582-9ea4-f57002824093 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.440154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1127/2229] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:03:57.445034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb05ae-1561-410a-bfed-8db2ec4c6d08 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.457805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb05ae-1561-410a-bfed-8db2ec4c6d08 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:57.463164 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f79332-b769-4582-9ea4-f57002824093 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.464855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f19b6-d1f7-4ba4-8fc4-84a83903f758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:57.464991 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c2f79332-b769-4582-9ea4-f57002824093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.465865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f19b6-d1f7-4ba4-8fc4-84a83903f758 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.466283 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f19b6-d1f7-4ba4-8fc4-84a83903f758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.466634 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f19b6-d1f7-4ba4-8fc4-84a83903f758 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.468168 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f19b6-d1f7-4ba4-8fc4-84a83903f758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.471601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f19b6-d1f7-4ba4-8fc4-84a83903f758 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.486173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f79332-b769-4582-9ea4-f57002824093 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:57.493592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85226db-b082-4b0f-89cb-0dd8eca52e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:57.494409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85226db-b082-4b0f-89cb-0dd8eca52e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.495251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85226db-b082-4b0f-89cb-0dd8eca52e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.495753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85226db-b082-4b0f-89cb-0dd8eca52e8d admin admin] PATH_INFO: `/v3/users/516ee32b9f6a46d1a4e3690fc036ebe8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.497939 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85226db-b082-4b0f-89cb-0dd8eca52e8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=516ee32b9f6a46d1a4e3690fc036ebe8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.501628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85226db-b082-4b0f-89cb-0dd8eca52e8d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.527578 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1103/2230] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:57 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:57.532373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f19b6-d1f7-4ba4-8fc4-84a83903f758 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.573121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c85226db-b082-4b0f-89cb-0dd8eca52e8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516ee32b9f6a46d1a4e3690fc036ebe8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:57.573914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c85226db-b082-4b0f-89cb-0dd8eca52e8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516ee32b9f6a46d1a4e3690fc036ebe8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:57.588650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c85226db-b082-4b0f-89cb-0dd8eca52e8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516ee32b9f6a46d1a4e3690fc036ebe8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:57.600098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c85226db-b082-4b0f-89cb-0dd8eca52e8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516ee32b9f6a46d1a4e3690fc036ebe8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:57.609539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c85226db-b082-4b0f-89cb-0dd8eca52e8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516ee32b9f6a46d1a4e3690fc036ebe8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:57.612369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f19b6-d1f7-4ba4-8fc4-84a83903f758 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:57.617387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735ccba-9266-47b3-b3da-b1ac49d8e96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:57.617550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1128/2231] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:57 2022] DELETE /identity/v3/users/516ee32b9f6a46d1a4e3690fc036ebe8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:57.617996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735ccba-9266-47b3-b3da-b1ac49d8e96b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.618273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735ccba-9266-47b3-b3da-b1ac49d8e96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.618547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735ccba-9266-47b3-b3da-b1ac49d8e96b admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.619205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735ccba-9266-47b3-b3da-b1ac49d8e96b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.621409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735ccba-9266-47b3-b3da-b1ac49d8e96b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.621770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619000f-a8c7-4309-a823-3346681ee596 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.622060 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619000f-a8c7-4309-a823-3346681ee596 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.622333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619000f-a8c7-4309-a823-3346681ee596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.646008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5619000f-a8c7-4309-a823-3346681ee596 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:57.647278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5619000f-a8c7-4309-a823-3346681ee596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.651220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1104/2232] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:57 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:03:57.652503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735ccba-9266-47b3-b3da-b1ac49d8e96b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.654258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0735ccba-9266-47b3-b3da-b1ac49d8e96b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.666495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735ccba-9266-47b3-b3da-b1ac49d8e96b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:57.684121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:57.703439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:57.718756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1129/2233] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:03:57.728310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619000f-a8c7-4309-a823-3346681ee596 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.729762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5619000f-a8c7-4309-a823-3346681ee596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.729891 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73ded2-f8c2-4434-9416-55d5b7997088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:57.730427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73ded2-f8c2-4434-9416-55d5b7997088 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.730705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73ded2-f8c2-4434-9416-55d5b7997088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.730979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73ded2-f8c2-4434-9416-55d5b7997088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.731949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73ded2-f8c2-4434-9416-55d5b7997088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.733534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73ded2-f8c2-4434-9416-55d5b7997088 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.743441 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1105/2234] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:03:57.747870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73ded2-f8c2-4434-9416-55d5b7997088 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.757075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73ded2-f8c2-4434-9416-55d5b7997088 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:57.761697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f492dc-eb9b-483d-a72d-d576d7a67cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:57.762348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f492dc-eb9b-483d-a72d-d576d7a67cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.762702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f492dc-eb9b-483d-a72d-d576d7a67cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.763011 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f492dc-eb9b-483d-a72d-d576d7a67cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.764182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f492dc-eb9b-483d-a72d-d576d7a67cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.765735 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f492dc-eb9b-483d-a72d-d576d7a67cbc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.775396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1106/2235] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:03:57.779551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f492dc-eb9b-483d-a72d-d576d7a67cbc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.788813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f492dc-eb9b-483d-a72d-d576d7a67cbc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:57.789716 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619000f-a8c7-4309-a823-3346681ee596 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:57.793320 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f344b-f75b-4de8-819f-d926b4374f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:57.793474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce4f9-b5cb-4384-9a75-ddba1353da4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:57.793948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f344b-f75b-4de8-819f-d926b4374f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.794086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce4f9-b5cb-4384-9a75-ddba1353da4e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.794302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce4f9-b5cb-4384-9a75-ddba1353da4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.794422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f344b-f75b-4de8-819f-d926b4374f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.794508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce4f9-b5cb-4384-9a75-ddba1353da4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.794587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f344b-f75b-4de8-819f-d926b4374f41 admin admin] PATH_INFO: `/v3/projects/874cd7b33a4c4c5699400dbbf0f41d5b/users/a8a2231fd0df4fc495b80ebeff7d6038/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.804062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ce4f9-b5cb-4384-9a75-ddba1353da4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.805481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ce4f9-b5cb-4384-9a75-ddba1353da4e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.809644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f344b-f75b-4de8-819f-d926b4374f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=874cd7b33a4c4c5699400dbbf0f41d5b, user_id=a8a2231fd0df4fc495b80ebeff7d6038, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.812858 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f344b-f75b-4de8-819f-d926b4374f41 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.815466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1130/2236] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:03:57.828629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1107/2237] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:57 2022] PUT /identity/v3/projects/874cd7b33a4c4c5699400dbbf0f41d5b/users/a8a2231fd0df4fc495b80ebeff7d6038/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:57.839024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8732b7-8c83-44d0-a2b6-671758020114 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.839280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8732b7-8c83-44d0-a2b6-671758020114 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.839559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8732b7-8c83-44d0-a2b6-671758020114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.869370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3c8732b7-8c83-44d0-a2b6-671758020114 None None] MFA Rules not processed for user `a8a2231fd0df4fc495b80ebeff7d6038`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:57.871172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3c8732b7-8c83-44d0-a2b6-671758020114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.944039 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1131/2238] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:03:57.956215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f344b-f75b-4de8-819f-d926b4374f41 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.957899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b63f344b-f75b-4de8-819f-d926b4374f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:58.042515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f344b-f75b-4de8-819f-d926b4374f41 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:58.048082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408f599-33e2-4e7e-8819-0c792a332d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:58.049134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408f599-33e2-4e7e-8819-0c792a332d15 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:58.049600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408f599-33e2-4e7e-8819-0c792a332d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:58.050113 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408f599-33e2-4e7e-8819-0c792a332d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:58.063219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408f599-33e2-4e7e-8819-0c792a332d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:58.064973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408f599-33e2-4e7e-8819-0c792a332d15 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:58.077172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1108/2239] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:58.140198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8732b7-8c83-44d0-a2b6-671758020114 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:58.143216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3c8732b7-8c83-44d0-a2b6-671758020114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:58.252690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8732b7-8c83-44d0-a2b6-671758020114 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:58.270915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:58.271837 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:58.272184 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:58.272652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] PATH_INFO: `/v3/projects/c7ae59ef02f640a5988fc8b1debfbc59` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:58.274015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ae59ef02f640a5988fc8b1debfbc59)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:58.277168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:58.317036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:58.336016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ae59ef02f640a5988fc8b1debfbc59', 'request_id': 'req-9e4dd4ff-5e7f-4177-935e-48b970f6edae'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.350728 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ae59ef02f640a5988fc8b1debfbc59', 'request_id': 'req-9e4dd4ff-5e7f-4177-935e-48b970f6edae'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.362069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ae59ef02f640a5988fc8b1debfbc59', 'request_id': 'req-9e4dd4ff-5e7f-4177-935e-48b970f6edae'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.379287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] The token cache is being invalidate because project c7ae59ef02f640a5988fc8b1debfbc59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:58.379587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.380254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.381864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1132/2240] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:58 2022] DELETE /identity/v3/projects/c7ae59ef02f640a5988fc8b1debfbc59 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:58.386579 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408f599-33e2-4e7e-8819-0c792a332d15 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:58.399984 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408f599-33e2-4e7e-8819-0c792a332d15 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:58.492828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c917922-241f-44c3-8d11-c72ac565ccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:58.493468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c917922-241f-44c3-8d11-c72ac565ccb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:58.493741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c917922-241f-44c3-8d11-c72ac565ccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:58.494008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c917922-241f-44c3-8d11-c72ac565ccb4 admin admin] PATH_INFO: `/v3/users/8bb5bcd3916b4cfa810169687a30d267` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:58.495264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c917922-241f-44c3-8d11-c72ac565ccb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bb5bcd3916b4cfa810169687a30d267)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:58.497587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c917922-241f-44c3-8d11-c72ac565ccb4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:58.528423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:58.557775 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c917922-241f-44c3-8d11-c72ac565ccb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bb5bcd3916b4cfa810169687a30d267'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.573968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c917922-241f-44c3-8d11-c72ac565ccb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bb5bcd3916b4cfa810169687a30d267'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.574771 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c917922-241f-44c3-8d11-c72ac565ccb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bb5bcd3916b4cfa810169687a30d267'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.575474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c917922-241f-44c3-8d11-c72ac565ccb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bb5bcd3916b4cfa810169687a30d267'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.583185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c917922-241f-44c3-8d11-c72ac565ccb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bb5bcd3916b4cfa810169687a30d267'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.598361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1109/2241] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:58 2022] DELETE /identity/v3/users/8bb5bcd3916b4cfa810169687a30d267 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:58.884570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:58.886065 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:58.900533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dd4ff-5e7f-4177-935e-48b970f6edae admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:58.969719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:58.970355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:58.970634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:58.970906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] PATH_INFO: `/v3/projects/395327af56004c41b6d27babf808a320` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:58.972043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=395327af56004c41b6d27babf808a320)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:58.974015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.019949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '395327af56004c41b6d27babf808a320', 'request_id': 'req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:59.026738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '395327af56004c41b6d27babf808a320', 'request_id': 'req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:59.032879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '395327af56004c41b6d27babf808a320', 'request_id': 'req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:59.043175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] The token cache is being invalidate because project 395327af56004c41b6d27babf808a320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:59.043470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:59.044099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a0c1d260-74cb-4b28-8a90-2e779bb5ef67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:59.046093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1133/2242] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:58 2022] DELETE /identity/v3/projects/395327af56004c41b6d27babf808a320 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:59.051026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f17783-62a0-4ea7-b1cb-5f3250813387 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.051333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f17783-62a0-4ea7-b1cb-5f3250813387 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.051610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f17783-62a0-4ea7-b1cb-5f3250813387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.078069 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d2f17783-62a0-4ea7-b1cb-5f3250813387 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:59.079804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d2f17783-62a0-4ea7-b1cb-5f3250813387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:59.142205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1110/2243] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:59.149962 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f8609-56ec-4564-bcef-1c738fa39b8a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.150332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f8609-56ec-4564-bcef-1c738fa39b8a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.150521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f8609-56ec-4564-bcef-1c738fa39b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.171661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-212f8609-56ec-4564-bcef-1c738fa39b8a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:59.172862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-212f8609-56ec-4564-bcef-1c738fa39b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:59.180790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1134/2244] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:59.184963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f17783-62a0-4ea7-b1cb-5f3250813387 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.194916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f17783-62a0-4ea7-b1cb-5f3250813387 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.200632 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f57bc-2dd8-480c-b65e-9ee7803bc649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.201238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f57bc-2dd8-480c-b65e-9ee7803bc649 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.201524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f57bc-2dd8-480c-b65e-9ee7803bc649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.201802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f57bc-2dd8-480c-b65e-9ee7803bc649 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.202452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f57bc-2dd8-480c-b65e-9ee7803bc649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:03:59.202756 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f57bc-2dd8-480c-b65e-9ee7803bc649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.204392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f57bc-2dd8-480c-b65e-9ee7803bc649 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.215817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1111/2245] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:03:59.220998 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f8609-56ec-4564-bcef-1c738fa39b8a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.230296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f8609-56ec-4564-bcef-1c738fa39b8a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.236177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295a71e-ce5c-4aa0-990f-1800f5f28a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.236771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295a71e-ce5c-4aa0-990f-1800f5f28a7c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.237053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295a71e-ce5c-4aa0-990f-1800f5f28a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.237326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295a71e-ce5c-4aa0-990f-1800f5f28a7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.237954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295a71e-ce5c-4aa0-990f-1800f5f28a7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.239893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295a71e-ce5c-4aa0-990f-1800f5f28a7c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.267338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1135/2246] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:59 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:59.272096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f57bc-2dd8-480c-b65e-9ee7803bc649 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.283665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f57bc-2dd8-480c-b65e-9ee7803bc649 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.342691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39356356-7b93-472e-a097-6f8802c3eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.343336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39356356-7b93-472e-a097-6f8802c3eff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.343620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39356356-7b93-472e-a097-6f8802c3eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.343892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39356356-7b93-472e-a097-6f8802c3eff6 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.344521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39356356-7b93-472e-a097-6f8802c3eff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.346427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39356356-7b93-472e-a097-6f8802c3eff6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.369431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1112/2247] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:59 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:59.373723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295a71e-ce5c-4aa0-990f-1800f5f28a7c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.384473 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295a71e-ce5c-4aa0-990f-1800f5f28a7c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.389908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619b0b7-1bd9-4f80-8d2d-0bec489051a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.390736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619b0b7-1bd9-4f80-8d2d-0bec489051a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.391107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619b0b7-1bd9-4f80-8d2d-0bec489051a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.391387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619b0b7-1bd9-4f80-8d2d-0bec489051a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.392033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619b0b7-1bd9-4f80-8d2d-0bec489051a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.394002 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619b0b7-1bd9-4f80-8d2d-0bec489051a6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.405659 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1136/2248] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:59.409871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39356356-7b93-472e-a097-6f8802c3eff6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.420978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39356356-7b93-472e-a097-6f8802c3eff6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.426106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a0ae2-22d2-4139-aec1-00817bff6de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.426659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a0ae2-22d2-4139-aec1-00817bff6de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.426939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a0ae2-22d2-4139-aec1-00817bff6de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.427246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a0ae2-22d2-4139-aec1-00817bff6de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.427893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a0ae2-22d2-4139-aec1-00817bff6de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.432996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a0ae2-22d2-4139-aec1-00817bff6de9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.447070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1113/2249] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:03:59.451952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619b0b7-1bd9-4f80-8d2d-0bec489051a6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.462156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619b0b7-1bd9-4f80-8d2d-0bec489051a6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.467745 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89116f21-d5e0-4ea9-9381-bff1ff93ebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.468396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89116f21-d5e0-4ea9-9381-bff1ff93ebab admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.468695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89116f21-d5e0-4ea9-9381-bff1ff93ebab admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.469042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89116f21-d5e0-4ea9-9381-bff1ff93ebab admin admin] PATH_INFO: `/v3/projects/058bf9ce2b414c4ca567e009e11f8e59/users/df917b7607a84551b757b6542304d1b5/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.471361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:59.487394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89116f21-d5e0-4ea9-9381-bff1ff93ebab admin admin] RBAC: Authorizing `identity:create_grant(project_id=058bf9ce2b414c4ca567e009e11f8e59, user_id=df917b7607a84551b757b6542304d1b5, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.490093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89116f21-d5e0-4ea9-9381-bff1ff93ebab admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.505364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1137/2250] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:59 2022] PUT /identity/v3/projects/058bf9ce2b414c4ca567e009e11f8e59/users/df917b7607a84551b757b6542304d1b5/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:59.516284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33ca7f-bad3-4173-869b-fc92d25c7538 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.516573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33ca7f-bad3-4173-869b-fc92d25c7538 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.516853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33ca7f-bad3-4173-869b-fc92d25c7538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.537217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:59.548364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bd33ca7f-bad3-4173-869b-fc92d25c7538 None None] MFA Rules not processed for user `df917b7607a84551b757b6542304d1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:59.549570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bd33ca7f-bad3-4173-869b-fc92d25c7538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:59.617597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1114/2251] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:03:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:59.622431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89116f21-d5e0-4ea9-9381-bff1ff93ebab admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.690652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89116f21-d5e0-4ea9-9381-bff1ff93ebab admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.695635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0a344-5a7f-49ed-b740-777c8a57b614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.696238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0a344-5a7f-49ed-b740-777c8a57b614 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.696515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0a344-5a7f-49ed-b740-777c8a57b614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.696913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0a344-5a7f-49ed-b740-777c8a57b614 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.697546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0a344-5a7f-49ed-b740-777c8a57b614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.699453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0a344-5a7f-49ed-b740-777c8a57b614 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.727317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1138/2252] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:59 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:59.731313 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33ca7f-bad3-4173-869b-fc92d25c7538 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.742782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33ca7f-bad3-4173-869b-fc92d25c7538 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.800644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da91bfb-be34-46e2-9868-cf99ffc533df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.801255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da91bfb-be34-46e2-9868-cf99ffc533df admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.801549 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da91bfb-be34-46e2-9868-cf99ffc533df admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.801820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da91bfb-be34-46e2-9868-cf99ffc533df admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.802607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da91bfb-be34-46e2-9868-cf99ffc533df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.804589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da91bfb-be34-46e2-9868-cf99ffc533df admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.826805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1115/2253] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:59 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:59.830715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0a344-5a7f-49ed-b740-777c8a57b614 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.841917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0a344-5a7f-49ed-b740-777c8a57b614 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.846525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf9b50-b40f-48ef-a004-b365bca44e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.847152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf9b50-b40f-48ef-a004-b365bca44e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.847438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf9b50-b40f-48ef-a004-b365bca44e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.847713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf9b50-b40f-48ef-a004-b365bca44e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.848350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf9b50-b40f-48ef-a004-b365bca44e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.849980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf9b50-b40f-48ef-a004-b365bca44e2a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.860617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1139/2254] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:59.864671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da91bfb-be34-46e2-9868-cf99ffc533df admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.875425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da91bfb-be34-46e2-9868-cf99ffc533df admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.880089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f4f32-6e53-4b1b-b3d1-9dff44cadce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.880727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f4f32-6e53-4b1b-b3d1-9dff44cadce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.881003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f4f32-6e53-4b1b-b3d1-9dff44cadce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.881276 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f4f32-6e53-4b1b-b3d1-9dff44cadce6 admin admin] PATH_INFO: `/v3/projects/d437a1101ed44a5b91231b44f82b7cec/users/f87411f8be0e47b38c00398d80eae232/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.896603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f4f32-6e53-4b1b-b3d1-9dff44cadce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d437a1101ed44a5b91231b44f82b7cec, user_id=f87411f8be0e47b38c00398d80eae232, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.899281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f4f32-6e53-4b1b-b3d1-9dff44cadce6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.912300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1116/2255] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:59 2022] PUT /identity/v3/projects/d437a1101ed44a5b91231b44f82b7cec/users/f87411f8be0e47b38c00398d80eae232/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:03:59.916156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf9b50-b40f-48ef-a004-b365bca44e2a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.926474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf9b50-b40f-48ef-a004-b365bca44e2a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.986933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf55f3-3379-40de-a861-930554cd7bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.987588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf55f3-3379-40de-a861-930554cd7bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.987869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf55f3-3379-40de-a861-930554cd7bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.988143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf55f3-3379-40de-a861-930554cd7bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.988782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf55f3-3379-40de-a861-930554cd7bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.990358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf55f3-3379-40de-a861-930554cd7bf0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:00.000499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1140/2256] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:00.004600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f4f32-6e53-4b1b-b3d1-9dff44cadce6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:00.017398 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f4f32-6e53-4b1b-b3d1-9dff44cadce6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:00.021953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66650cb-fc02-427b-b29b-168a5c74e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:00.022588 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66650cb-fc02-427b-b29b-168a5c74e9d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:00.022864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66650cb-fc02-427b-b29b-168a5c74e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:00.023161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66650cb-fc02-427b-b29b-168a5c74e9d2 admin admin] PATH_INFO: `/v3/projects/d437a1101ed44a5b91231b44f82b7cec/users/f87411f8be0e47b38c00398d80eae232/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:00.025060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66650cb-fc02-427b-b29b-168a5c74e9d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d437a1101ed44a5b91231b44f82b7cec, user_id=f87411f8be0e47b38c00398d80eae232, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:00.027709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66650cb-fc02-427b-b29b-168a5c74e9d2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:00.037769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1117/2257] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:00 2022] PUT /identity/v3/projects/d437a1101ed44a5b91231b44f82b7cec/users/f87411f8be0e47b38c00398d80eae232/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:04:00.048912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bc71d-6164-492a-8628-3648b3d2fb65 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:00.049190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bc71d-6164-492a-8628-3648b3d2fb65 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:00.049488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bc71d-6164-492a-8628-3648b3d2fb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:00.079508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2b9bc71d-6164-492a-8628-3648b3d2fb65 None None] MFA Rules not processed for user `f87411f8be0e47b38c00398d80eae232`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:00.080726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2b9bc71d-6164-492a-8628-3648b3d2fb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:00.125058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:00.154795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1141/2258] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:00.166168 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66650cb-fc02-427b-b29b-168a5c74e9d2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:00.167813 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f66650cb-fc02-427b-b29b-168a5c74e9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:00.224163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66650cb-fc02-427b-b29b-168a5c74e9d2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:00.228127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3edc0a-860d-451b-b739-c1a632fbd807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:00.228712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3edc0a-860d-451b-b739-c1a632fbd807 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:00.229012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3edc0a-860d-451b-b739-c1a632fbd807 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:00.229280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3edc0a-860d-451b-b739-c1a632fbd807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:00.235836 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:00.239619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3edc0a-860d-451b-b739-c1a632fbd807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:00.242523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3edc0a-860d-451b-b739-c1a632fbd807 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:00.252722 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1118/2259] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:00.317441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bc71d-6164-492a-8628-3648b3d2fb65 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:00.381142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bc71d-6164-492a-8628-3648b3d2fb65 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:00.385644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:00.386264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:00.386541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:00.386815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] PATH_INFO: `/v3/users/df917b7607a84551b757b6542304d1b5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:00.388045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df917b7607a84551b757b6542304d1b5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:00.390558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:00.438020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df917b7607a84551b757b6542304d1b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:00.438588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df917b7607a84551b757b6542304d1b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:00.451551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df917b7607a84551b757b6542304d1b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:00.460875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df917b7607a84551b757b6542304d1b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:00.476244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df917b7607a84551b757b6542304d1b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:00.483438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1142/2260] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:04:00 2022] DELETE /identity/v3/users/df917b7607a84551b757b6542304d1b5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:00.488208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b8aa-63fa-4f99-8c02-aa36c88e6424 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:00.488828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b8aa-63fa-4f99-8c02-aa36c88e6424 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:00.489083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b8aa-63fa-4f99-8c02-aa36c88e6424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:00.513721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6074b8aa-63fa-4f99-8c02-aa36c88e6424 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:00.514956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6074b8aa-63fa-4f99-8c02-aa36c88e6424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:00.574089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1119/2261] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:00.584187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:00.585602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:00.643736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef354664-efb7-4808-ac3f-66a90a1ae760 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:00.647358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ff59e-525b-4d5a-ae0a-a570a6c34512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:00.647957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ff59e-525b-4d5a-ae0a-a570a6c34512 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:00.648238 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ff59e-525b-4d5a-ae0a-a570a6c34512 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:00.648516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ff59e-525b-4d5a-ae0a-a570a6c34512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:00.657865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ff59e-525b-4d5a-ae0a-a570a6c34512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:00.659316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ff59e-525b-4d5a-ae0a-a570a6c34512 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:00.664203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:00.669326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1143/2262] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:04:00.960766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074b8aa-63fa-4f99-8c02-aa36c88e6424 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:00.962308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6074b8aa-63fa-4f99-8c02-aa36c88e6424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:00.975773 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074b8aa-63fa-4f99-8c02-aa36c88e6424 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:00.980647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:00.981398 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:00.981589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:00.981867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] PATH_INFO: `/v3/projects/058bf9ce2b414c4ca567e009e11f8e59` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:00.983657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=058bf9ce2b414c4ca567e009e11f8e59)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:00.985571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:01.033917 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '058bf9ce2b414c4ca567e009e11f8e59', 'request_id': 'req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.040139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '058bf9ce2b414c4ca567e009e11f8e59', 'request_id': 'req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.045710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '058bf9ce2b414c4ca567e009e11f8e59', 'request_id': 'req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.058046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] The token cache is being invalidate because project 058bf9ce2b414c4ca567e009e11f8e59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:04:01.058337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.059012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.060563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1120/2263] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:04:00 2022] DELETE /identity/v3/projects/058bf9ce2b414c4ca567e009e11f8e59 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:01.064534 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ff59e-525b-4d5a-ae0a-a570a6c34512 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:01.074662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ff59e-525b-4d5a-ae0a-a570a6c34512 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:01.129770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3768f2-8419-4f51-910d-ebea180c42e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:01.130385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3768f2-8419-4f51-910d-ebea180c42e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:01.130663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3768f2-8419-4f51-910d-ebea180c42e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:01.130934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3768f2-8419-4f51-910d-ebea180c42e2 admin admin] PATH_INFO: `/v3/users/f87411f8be0e47b38c00398d80eae232` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:01.132064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3768f2-8419-4f51-910d-ebea180c42e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87411f8be0e47b38c00398d80eae232)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:01.134041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3768f2-8419-4f51-910d-ebea180c42e2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:01.183776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9b3768f2-8419-4f51-910d-ebea180c42e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87411f8be0e47b38c00398d80eae232'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.184423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9b3768f2-8419-4f51-910d-ebea180c42e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87411f8be0e47b38c00398d80eae232'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.196513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9b3768f2-8419-4f51-910d-ebea180c42e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87411f8be0e47b38c00398d80eae232'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.204488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9b3768f2-8419-4f51-910d-ebea180c42e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87411f8be0e47b38c00398d80eae232'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.211416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9b3768f2-8419-4f51-910d-ebea180c42e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87411f8be0e47b38c00398d80eae232'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.218253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1144/2264] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:04:01 2022] DELETE /identity/v3/users/f87411f8be0e47b38c00398d80eae232 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:01.488212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:01.489675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:01.504465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eda4d1-a10e-4dd5-9974-5c7d3bb6dfa2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:01.565005 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:01.565635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:01.565944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:01.566183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] PATH_INFO: `/v3/projects/d437a1101ed44a5b91231b44f82b7cec` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:01.567321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d437a1101ed44a5b91231b44f82b7cec)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:01.569464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:01.590098 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:01.618471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd437a1101ed44a5b91231b44f82b7cec', 'request_id': 'req-d72b5f49-5799-4477-b26c-a62340ccc45d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.624820 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd437a1101ed44a5b91231b44f82b7cec', 'request_id': 'req-d72b5f49-5799-4477-b26c-a62340ccc45d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.630815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd437a1101ed44a5b91231b44f82b7cec', 'request_id': 'req-d72b5f49-5799-4477-b26c-a62340ccc45d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.645022 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] The token cache is being invalidate because project d437a1101ed44a5b91231b44f82b7cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:04:01.645310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.646396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d72b5f49-5799-4477-b26c-a62340ccc45d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:01.647919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1121/2265] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:04:01 2022] DELETE /identity/v3/projects/d437a1101ed44a5b91231b44f82b7cec => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:01.652922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69054f-5a23-4428-96c5-9ec059e366ca None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:01.653217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69054f-5a23-4428-96c5-9ec059e366ca None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:01.653498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69054f-5a23-4428-96c5-9ec059e366ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:01.674826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3c69054f-5a23-4428-96c5-9ec059e366ca None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:01.676082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3c69054f-5a23-4428-96c5-9ec059e366ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:01.739042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1145/2266] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:04:01.747166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42f084-1576-4391-9f94-32f214bdc0c7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:01.747451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42f084-1576-4391-9f94-32f214bdc0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:01.748238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42f084-1576-4391-9f94-32f214bdc0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:01.769558 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-de42f084-1576-4391-9f94-32f214bdc0c7 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:01.770773 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-de42f084-1576-4391-9f94-32f214bdc0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:01.779105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1122/2267] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:01.783086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69054f-5a23-4428-96c5-9ec059e366ca None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:01.793327 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69054f-5a23-4428-96c5-9ec059e366ca None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:01.799214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743f093-997a-493a-ad56-044c074d8038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:01.799982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743f093-997a-493a-ad56-044c074d8038 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:01.800400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743f093-997a-493a-ad56-044c074d8038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:01.800963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743f093-997a-493a-ad56-044c074d8038 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:01.801773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743f093-997a-493a-ad56-044c074d8038 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:04:01.802415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743f093-997a-493a-ad56-044c074d8038 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:01.804176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743f093-997a-493a-ad56-044c074d8038 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:01.816980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1146/2268] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:04:01.821777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42f084-1576-4391-9f94-32f214bdc0c7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:01.831749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42f084-1576-4391-9f94-32f214bdc0c7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:01.836538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87123b6e-8d01-4ef4-90c3-726c81ce81a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:01.837164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87123b6e-8d01-4ef4-90c3-726c81ce81a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:01.837444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87123b6e-8d01-4ef4-90c3-726c81ce81a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:01.837757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87123b6e-8d01-4ef4-90c3-726c81ce81a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:01.838390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87123b6e-8d01-4ef4-90c3-726c81ce81a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:01.840339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87123b6e-8d01-4ef4-90c3-726c81ce81a9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:01.868115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1123/2269] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:04:01 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:04:01.872299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743f093-997a-493a-ad56-044c074d8038 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:01.882831 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743f093-997a-493a-ad56-044c074d8038 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:01.886364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:01.940960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca605181-6e45-43f2-91de-d39e27d5dc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:01.941556 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca605181-6e45-43f2-91de-d39e27d5dc3c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:01.941833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca605181-6e45-43f2-91de-d39e27d5dc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:01.942101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca605181-6e45-43f2-91de-d39e27d5dc3c admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:01.942730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca605181-6e45-43f2-91de-d39e27d5dc3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:01.944930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca605181-6e45-43f2-91de-d39e27d5dc3c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:01.969156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1147/2270] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:04:01 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:04:01.973371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87123b6e-8d01-4ef4-90c3-726c81ce81a9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:01.984375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87123b6e-8d01-4ef4-90c3-726c81ce81a9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:01.988838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad846cf4-6d2f-4491-ab9c-fc53020b0d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:01.989424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad846cf4-6d2f-4491-ab9c-fc53020b0d53 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:01.989702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad846cf4-6d2f-4491-ab9c-fc53020b0d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:01.989971 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad846cf4-6d2f-4491-ab9c-fc53020b0d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:01.990605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad846cf4-6d2f-4491-ab9c-fc53020b0d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:01.992219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad846cf4-6d2f-4491-ab9c-fc53020b0d53 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:02.002951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1124/2271] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:02.006933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca605181-6e45-43f2-91de-d39e27d5dc3c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:02.017054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca605181-6e45-43f2-91de-d39e27d5dc3c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:02.022962 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60e994-91de-4312-974a-f56aa7a1692f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:02.023625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60e994-91de-4312-974a-f56aa7a1692f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.023901 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60e994-91de-4312-974a-f56aa7a1692f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.024170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60e994-91de-4312-974a-f56aa7a1692f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:02.024806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60e994-91de-4312-974a-f56aa7a1692f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:02.026637 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60e994-91de-4312-974a-f56aa7a1692f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:02.037574 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1148/2272] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:04:02.041781 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad846cf4-6d2f-4491-ab9c-fc53020b0d53 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:02.052556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad846cf4-6d2f-4491-ab9c-fc53020b0d53 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:02.057248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bd1b7-2750-4451-8abe-522baf3eca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:02.057898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bd1b7-2750-4451-8abe-522baf3eca0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.058181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bd1b7-2750-4451-8abe-522baf3eca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.058462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bd1b7-2750-4451-8abe-522baf3eca0f admin admin] PATH_INFO: `/v3/projects/bb364b49673642459ef4a069df42a157/users/d91e21fca19140c1b7de75e0a4dc8a8d/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:02.073762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bd1b7-2750-4451-8abe-522baf3eca0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb364b49673642459ef4a069df42a157, user_id=d91e21fca19140c1b7de75e0a4dc8a8d, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:02.076465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bd1b7-2750-4451-8abe-522baf3eca0f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:02.089361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1125/2273] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:02 2022] PUT /identity/v3/projects/bb364b49673642459ef4a069df42a157/users/d91e21fca19140c1b7de75e0a4dc8a8d/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:02.099524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc57b35-a102-4224-825f-b1d2f40bbe4e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.099966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc57b35-a102-4224-825f-b1d2f40bbe4e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.100078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc57b35-a102-4224-825f-b1d2f40bbe4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:02.132208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2fc57b35-a102-4224-825f-b1d2f40bbe4e None None] MFA Rules not processed for user `d91e21fca19140c1b7de75e0a4dc8a8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:02.133745 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2fc57b35-a102-4224-825f-b1d2f40bbe4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:02.203384 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1149/2274] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:02.207794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bd1b7-2750-4451-8abe-522baf3eca0f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:02.271017 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bd1b7-2750-4451-8abe-522baf3eca0f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:02.275641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967bf302-384b-42f7-a889-493426509ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:02.276235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bf302-384b-42f7-a889-493426509ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.276513 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bf302-384b-42f7-a889-493426509ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.276781 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bf302-384b-42f7-a889-493426509ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:02.277397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967bf302-384b-42f7-a889-493426509ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:02.280117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967bf302-384b-42f7-a889-493426509ed5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:02.306334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1126/2275] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:04:02 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:04:02.310441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc57b35-a102-4224-825f-b1d2f40bbe4e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:02.321362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc57b35-a102-4224-825f-b1d2f40bbe4e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:02.377646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789af34-dc25-48bd-87e4-438c20c6c3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:02.378249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789af34-dc25-48bd-87e4-438c20c6c3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.378524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789af34-dc25-48bd-87e4-438c20c6c3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.378790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789af34-dc25-48bd-87e4-438c20c6c3d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:02.379447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789af34-dc25-48bd-87e4-438c20c6c3d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:02.381327 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789af34-dc25-48bd-87e4-438c20c6c3d6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:02.401611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1150/2276] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:04:02 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:04:02.406243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967bf302-384b-42f7-a889-493426509ed5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:02.416358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967bf302-384b-42f7-a889-493426509ed5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:02.421665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad288d4-dad0-4c39-8ad2-ad5623df1bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:02.422255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad288d4-dad0-4c39-8ad2-ad5623df1bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.422532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad288d4-dad0-4c39-8ad2-ad5623df1bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.422802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad288d4-dad0-4c39-8ad2-ad5623df1bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:02.423471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad288d4-dad0-4c39-8ad2-ad5623df1bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:02.425039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad288d4-dad0-4c39-8ad2-ad5623df1bd6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:02.435970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1127/2277] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:02.440144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789af34-dc25-48bd-87e4-438c20c6c3d6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:02.446708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:02.450555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789af34-dc25-48bd-87e4-438c20c6c3d6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:02.456502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681090eb-d3a0-475e-b409-bcec0b084cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:02.457172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681090eb-d3a0-475e-b409-bcec0b084cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.457451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681090eb-d3a0-475e-b409-bcec0b084cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.457723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681090eb-d3a0-475e-b409-bcec0b084cc1 admin admin] PATH_INFO: `/v3/projects/252497deab5f4d4fb090687a76dcbdee/users/4e90ae9eeeec404c8c0b4f46db052c50/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:02.472760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681090eb-d3a0-475e-b409-bcec0b084cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=252497deab5f4d4fb090687a76dcbdee, user_id=4e90ae9eeeec404c8c0b4f46db052c50, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:02.475495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681090eb-d3a0-475e-b409-bcec0b084cc1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:02.488134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1151/2278] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:02 2022] PUT /identity/v3/projects/252497deab5f4d4fb090687a76dcbdee/users/4e90ae9eeeec404c8c0b4f46db052c50/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:02.492048 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad288d4-dad0-4c39-8ad2-ad5623df1bd6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:02.502500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad288d4-dad0-4c39-8ad2-ad5623df1bd6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:02.515305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:02.571540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6156786-e5d7-4389-91ad-dd8ffe06f06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:02.572063 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6156786-e5d7-4389-91ad-dd8ffe06f06f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.572340 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6156786-e5d7-4389-91ad-dd8ffe06f06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.572611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6156786-e5d7-4389-91ad-dd8ffe06f06f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:02.573275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6156786-e5d7-4389-91ad-dd8ffe06f06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:02.574852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6156786-e5d7-4389-91ad-dd8ffe06f06f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:02.585230 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1128/2279] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:02.590051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681090eb-d3a0-475e-b409-bcec0b084cc1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:02.600293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681090eb-d3a0-475e-b409-bcec0b084cc1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:02.605188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a6a24-4a5c-4e99-9f0b-1d9d246b6b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:02.605868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a6a24-4a5c-4e99-9f0b-1d9d246b6b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.606144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a6a24-4a5c-4e99-9f0b-1d9d246b6b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.606413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a6a24-4a5c-4e99-9f0b-1d9d246b6b3c admin admin] PATH_INFO: `/v3/projects/252497deab5f4d4fb090687a76dcbdee/users/4e90ae9eeeec404c8c0b4f46db052c50/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:02.609072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a6a24-4a5c-4e99-9f0b-1d9d246b6b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=252497deab5f4d4fb090687a76dcbdee, user_id=4e90ae9eeeec404c8c0b4f46db052c50, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:02.611757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a6a24-4a5c-4e99-9f0b-1d9d246b6b3c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:02.622061 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1152/2280] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:02 2022] PUT /identity/v3/projects/252497deab5f4d4fb090687a76dcbdee/users/4e90ae9eeeec404c8c0b4f46db052c50/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:02.634175 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83027938-f701-45a7-b06d-04a3b08aeb94 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.634466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83027938-f701-45a7-b06d-04a3b08aeb94 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.634746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83027938-f701-45a7-b06d-04a3b08aeb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:02.662869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-83027938-f701-45a7-b06d-04a3b08aeb94 None None] MFA Rules not processed for user `4e90ae9eeeec404c8c0b4f46db052c50`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:02.664167 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-83027938-f701-45a7-b06d-04a3b08aeb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:02.735691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1129/2281] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:02.746890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a6a24-4a5c-4e99-9f0b-1d9d246b6b3c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:02.749221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-484a6a24-4a5c-4e99-9f0b-1d9d246b6b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:02.816508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a6a24-4a5c-4e99-9f0b-1d9d246b6b3c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:02.820424 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a285f-f9bd-4736-95df-698cbda4427a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:02.821032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a285f-f9bd-4736-95df-698cbda4427a service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.821309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a285f-f9bd-4736-95df-698cbda4427a service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.821659 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a285f-f9bd-4736-95df-698cbda4427a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:02.831088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a285f-f9bd-4736-95df-698cbda4427a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:02.833366 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a285f-f9bd-4736-95df-698cbda4427a service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:02.849531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1153/2282] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:02.947662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83027938-f701-45a7-b06d-04a3b08aeb94 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:02.960925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83027938-f701-45a7-b06d-04a3b08aeb94 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:02.964745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd6dc3-d5bd-48ba-aa5b-3a8946b2a4a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:02.965364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd6dc3-d5bd-48ba-aa5b-3a8946b2a4a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:02.965641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd6dc3-d5bd-48ba-aa5b-3a8946b2a4a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:02.965937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd6dc3-d5bd-48ba-aa5b-3a8946b2a4a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:03.037576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd6dc3-d5bd-48ba-aa5b-3a8946b2a4a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:03.039306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd6dc3-d5bd-48ba-aa5b-3a8946b2a4a7 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:03.048973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1130/2283] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:05.356013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b60eb-d318-4da2-9444-c522eeddeeec None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:05.356695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b60eb-d318-4da2-9444-c522eeddeeec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:05.356695 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b60eb-d318-4da2-9444-c522eeddeeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:05.377675 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f93b60eb-d318-4da2-9444-c522eeddeeec None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:05.378925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f93b60eb-d318-4da2-9444-c522eeddeeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:05.440720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1154/2284] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:05.486803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd6dc3-d5bd-48ba-aa5b-3a8946b2a4a7 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:05.488356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-92cd6dc3-d5bd-48ba-aa5b-3a8946b2a4a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:05.533125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:05.565707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd6dc3-d5bd-48ba-aa5b-3a8946b2a4a7 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:05.571208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2114aa-24b8-4297-acc8-07e363c6c294 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:05.571936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2114aa-24b8-4297-acc8-07e363c6c294 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:05.572216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2114aa-24b8-4297-acc8-07e363c6c294 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:05.572491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2114aa-24b8-4297-acc8-07e363c6c294 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:05.582351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2114aa-24b8-4297-acc8-07e363c6c294 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:05.583967 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2114aa-24b8-4297-acc8-07e363c6c294 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:05.592652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1131/2285] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:04:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:04:06.035557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b60eb-d318-4da2-9444-c522eeddeeec None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:06.037734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f93b60eb-d318-4da2-9444-c522eeddeeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:06.139831 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b60eb-d318-4da2-9444-c522eeddeeec None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:06.146146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:06.146854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:06.147187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:06.147467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] PATH_INFO: `/v3/users/a8a2231fd0df4fc495b80ebeff7d6038` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:06.148761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a2231fd0df4fc495b80ebeff7d6038)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:06.150877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:06.171706 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:06.217879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a2231fd0df4fc495b80ebeff7d6038'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:06.218952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a2231fd0df4fc495b80ebeff7d6038'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:06.233063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a2231fd0df4fc495b80ebeff7d6038'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:06.242411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a2231fd0df4fc495b80ebeff7d6038'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:06.251094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a2231fd0df4fc495b80ebeff7d6038'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:06.257441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1155/2286] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:04:06 2022] DELETE /identity/v3/users/a8a2231fd0df4fc495b80ebeff7d6038 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:06.262097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc943-4308-4a67-a316-53a1b712c41e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:06.262468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc943-4308-4a67-a316-53a1b712c41e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:06.262849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc943-4308-4a67-a316-53a1b712c41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:06.284019 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-658bc943-4308-4a67-a316-53a1b712c41e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:06.285575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-658bc943-4308-4a67-a316-53a1b712c41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:06.357469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1132/2287] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:06.364382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:06.366427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:06.434414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f57ba-f485-4bac-8884-7c428bd738cc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:06.438419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be376ad7-c1fd-44d6-b5b9-7c783c90c0fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:06.439232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be376ad7-c1fd-44d6-b5b9-7c783c90c0fd service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:06.439526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be376ad7-c1fd-44d6-b5b9-7c783c90c0fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:06.439801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be376ad7-c1fd-44d6-b5b9-7c783c90c0fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:06.451504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be376ad7-c1fd-44d6-b5b9-7c783c90c0fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:06.453277 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be376ad7-c1fd-44d6-b5b9-7c783c90c0fd service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:06.486363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1156/2288] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:06.867610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bc943-4308-4a67-a316-53a1b712c41e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:06.901452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-658bc943-4308-4a67-a316-53a1b712c41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:06.930086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bc943-4308-4a67-a316-53a1b712c41e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:06.960696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:06.965466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:06.974607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:06.977535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] PATH_INFO: `/v3/projects/874cd7b33a4c4c5699400dbbf0f41d5b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:06.980163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=874cd7b33a4c4c5699400dbbf0f41d5b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:06.985201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:07.108242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '874cd7b33a4c4c5699400dbbf0f41d5b', 'request_id': 'req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:07.116212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '874cd7b33a4c4c5699400dbbf0f41d5b', 'request_id': 'req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:07.145502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '874cd7b33a4c4c5699400dbbf0f41d5b', 'request_id': 'req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:07.172187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] The token cache is being invalidate because project 874cd7b33a4c4c5699400dbbf0f41d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:04:07.173377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:07.174293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a27c522b-bda5-404f-b4f7-b7a38ecbca8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:07.176425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1133/2289] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:04:06 2022] DELETE /identity/v3/projects/874cd7b33a4c4c5699400dbbf0f41d5b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:07.182019 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c9f03-400f-4e62-8d2d-732550509cb8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:07.182601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c9f03-400f-4e62-8d2d-732550509cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:07.183225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c9f03-400f-4e62-8d2d-732550509cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:07.212522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c41c9f03-400f-4e62-8d2d-732550509cb8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:07.213845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c41c9f03-400f-4e62-8d2d-732550509cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:07.305626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1157/2290] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:04:07.315286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df952d-8c35-4fa3-948a-884b16fa3bb4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:07.315894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df952d-8c35-4fa3-948a-884b16fa3bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:07.316266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df952d-8c35-4fa3-948a-884b16fa3bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:07.355407 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a4df952d-8c35-4fa3-948a-884b16fa3bb4 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:07.357190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a4df952d-8c35-4fa3-948a-884b16fa3bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:07.369765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1134/2291] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:07.373780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c9f03-400f-4e62-8d2d-732550509cb8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:07.386038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c9f03-400f-4e62-8d2d-732550509cb8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:07.391370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212e5e8-08ec-4d45-af12-6c18a3824960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:07.391981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212e5e8-08ec-4d45-af12-6c18a3824960 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:07.392248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212e5e8-08ec-4d45-af12-6c18a3824960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:07.392508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212e5e8-08ec-4d45-af12-6c18a3824960 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:07.393161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212e5e8-08ec-4d45-af12-6c18a3824960 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:04:07.393460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212e5e8-08ec-4d45-af12-6c18a3824960 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:07.397036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212e5e8-08ec-4d45-af12-6c18a3824960 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:07.421270 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1158/2292] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:04:07.421786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df952d-8c35-4fa3-948a-884b16fa3bb4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:07.434638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df952d-8c35-4fa3-948a-884b16fa3bb4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:07.447769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abf209-acf5-4b23-b009-af66f8d62e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:07.447769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abf209-acf5-4b23-b009-af66f8d62e7d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:07.447769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abf209-acf5-4b23-b009-af66f8d62e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:07.447769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abf209-acf5-4b23-b009-af66f8d62e7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:07.448085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abf209-acf5-4b23-b009-af66f8d62e7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:07.448085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abf209-acf5-4b23-b009-af66f8d62e7d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:07.503437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1135/2293] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:04:07 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:04:07.504001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212e5e8-08ec-4d45-af12-6c18a3824960 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:07.542941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212e5e8-08ec-4d45-af12-6c18a3824960 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:07.568900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:07.651197 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5436d-f9d2-4fdc-82ce-ca7fd5fd0586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:07.653153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5436d-f9d2-4fdc-82ce-ca7fd5fd0586 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:07.653545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5436d-f9d2-4fdc-82ce-ca7fd5fd0586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:07.654420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5436d-f9d2-4fdc-82ce-ca7fd5fd0586 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:07.655104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5436d-f9d2-4fdc-82ce-ca7fd5fd0586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:07.657499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5436d-f9d2-4fdc-82ce-ca7fd5fd0586 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:07.690689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1159/2294] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:04:07 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:04:07.695834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abf209-acf5-4b23-b009-af66f8d62e7d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:07.710740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abf209-acf5-4b23-b009-af66f8d62e7d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:07.722904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbae166-a7d4-4a6e-bd34-45807af7b8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:07.723629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbae166-a7d4-4a6e-bd34-45807af7b8cc admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:07.724507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbae166-a7d4-4a6e-bd34-45807af7b8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:07.724507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbae166-a7d4-4a6e-bd34-45807af7b8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:07.724889 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbae166-a7d4-4a6e-bd34-45807af7b8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:07.726405 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbae166-a7d4-4a6e-bd34-45807af7b8cc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:07.751306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1136/2295] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:07.754115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5436d-f9d2-4fdc-82ce-ca7fd5fd0586 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:07.765966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5436d-f9d2-4fdc-82ce-ca7fd5fd0586 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:07.771365 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d0ad1-e848-4ac3-81f0-106bb579ca95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:07.772082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d0ad1-e848-4ac3-81f0-106bb579ca95 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:07.772555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d0ad1-e848-4ac3-81f0-106bb579ca95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:07.772899 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d0ad1-e848-4ac3-81f0-106bb579ca95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:07.773863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d0ad1-e848-4ac3-81f0-106bb579ca95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:07.775694 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d0ad1-e848-4ac3-81f0-106bb579ca95 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:07.799133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1160/2296] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:04:07.800782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbae166-a7d4-4a6e-bd34-45807af7b8cc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:07.840999 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbae166-a7d4-4a6e-bd34-45807af7b8cc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:07.850213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eeeb24-6803-4f4e-b635-655083ba0e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:07.850884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eeeb24-6803-4f4e-b635-655083ba0e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:07.851227 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eeeb24-6803-4f4e-b635-655083ba0e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:07.851593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eeeb24-6803-4f4e-b635-655083ba0e92 admin admin] PATH_INFO: `/v3/projects/350221a9b0c14c70b92f48cae0d3a838/users/d0f11e586dc64a41a84248b4a4c894a9/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:07.866709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:07.881890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eeeb24-6803-4f4e-b635-655083ba0e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350221a9b0c14c70b92f48cae0d3a838, user_id=d0f11e586dc64a41a84248b4a4c894a9, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:07.887594 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eeeb24-6803-4f4e-b635-655083ba0e92 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:07.907505 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1137/2297] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:07 2022] PUT /identity/v3/projects/350221a9b0c14c70b92f48cae0d3a838/users/d0f11e586dc64a41a84248b4a4c894a9/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:07.923991 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94bef-0d97-4070-ad67-6df8b5c0c837 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:07.924536 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94bef-0d97-4070-ad67-6df8b5c0c837 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:07.924966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94bef-0d97-4070-ad67-6df8b5c0c837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:07.968129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b6d94bef-0d97-4070-ad67-6df8b5c0c837 None None] MFA Rules not processed for user `d0f11e586dc64a41a84248b4a4c894a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:07.969379 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b6d94bef-0d97-4070-ad67-6df8b5c0c837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:08.099160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1161/2298] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:08.112586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eeeb24-6803-4f4e-b635-655083ba0e92 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:08.114596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-08eeeb24-6803-4f4e-b635-655083ba0e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:08.194322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eeeb24-6803-4f4e-b635-655083ba0e92 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:08.201330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cc556-3feb-420c-a2d6-bbe92d948989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:08.201330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cc556-3feb-420c-a2d6-bbe92d948989 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:08.201330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cc556-3feb-420c-a2d6-bbe92d948989 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:08.201330 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cc556-3feb-420c-a2d6-bbe92d948989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:08.211641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cc556-3feb-420c-a2d6-bbe92d948989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:08.215040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cc556-3feb-420c-a2d6-bbe92d948989 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:08.226546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1138/2299] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:09.477453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d94bef-0d97-4070-ad67-6df8b5c0c837 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:09.478916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b6d94bef-0d97-4070-ad67-6df8b5c0c837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:09.551634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d94bef-0d97-4070-ad67-6df8b5c0c837 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:09.558629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d9d20-ff9a-4664-8a5e-9157a7683908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:09.559752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9d20-ff9a-4664-8a5e-9157a7683908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:09.560117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9d20-ff9a-4664-8a5e-9157a7683908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:09.560117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9d20-ff9a-4664-8a5e-9157a7683908 admin admin] PATH_INFO: `/v3/users/d91e21fca19140c1b7de75e0a4dc8a8d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:09.561486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d9d20-ff9a-4664-8a5e-9157a7683908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d91e21fca19140c1b7de75e0a4dc8a8d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:09.563451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d9d20-ff9a-4664-8a5e-9157a7683908 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:09.619162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:09.630976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a26d9d20-ff9a-4664-8a5e-9157a7683908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd91e21fca19140c1b7de75e0a4dc8a8d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:09.631902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a26d9d20-ff9a-4664-8a5e-9157a7683908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd91e21fca19140c1b7de75e0a4dc8a8d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:09.648594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a26d9d20-ff9a-4664-8a5e-9157a7683908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd91e21fca19140c1b7de75e0a4dc8a8d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:09.674755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcca34-21f1-4e9a-8a13-b925f2cfa97f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:09.675747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcca34-21f1-4e9a-8a13-b925f2cfa97f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:09.675747 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcca34-21f1-4e9a-8a13-b925f2cfa97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:09.682764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a26d9d20-ff9a-4664-8a5e-9157a7683908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd91e21fca19140c1b7de75e0a4dc8a8d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:09.701439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a26d9d20-ff9a-4664-8a5e-9157a7683908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd91e21fca19140c1b7de75e0a4dc8a8d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:09.704750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d3bcca34-21f1-4e9a-8a13-b925f2cfa97f None None] MFA Rules not processed for user `d0f11e586dc64a41a84248b4a4c894a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:09.706697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d3bcca34-21f1-4e9a-8a13-b925f2cfa97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:09.710131 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1162/2300] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:04:09 2022] DELETE /identity/v3/users/d91e21fca19140c1b7de75e0a4dc8a8d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:09.714331 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c3a1c-338b-494d-ad56-319aa4e8fcfa None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:09.714629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c3a1c-338b-494d-ad56-319aa4e8fcfa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:09.714928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c3a1c-338b-494d-ad56-319aa4e8fcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:09.745621 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ce1c3a1c-338b-494d-ad56-319aa4e8fcfa None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:09.748722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ce1c3a1c-338b-494d-ad56-319aa4e8fcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:09.798396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1139/2301] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:09.851028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1163/2302] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:09.859452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcca34-21f1-4e9a-8a13-b925f2cfa97f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:09.860967 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d3bcca34-21f1-4e9a-8a13-b925f2cfa97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:09.928784 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcca34-21f1-4e9a-8a13-b925f2cfa97f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:09.934300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce10a37-f0ea-4133-a5c6-fb1e8f8ceaec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:09.935569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce10a37-f0ea-4133-a5c6-fb1e8f8ceaec service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:09.935863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce10a37-f0ea-4133-a5c6-fb1e8f8ceaec service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:09.936152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce10a37-f0ea-4133-a5c6-fb1e8f8ceaec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:09.947849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce10a37-f0ea-4133-a5c6-fb1e8f8ceaec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:09.949460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce10a37-f0ea-4133-a5c6-fb1e8f8ceaec service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:09.961061 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1140/2303] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:04:10.965165 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c3a1c-338b-494d-ad56-319aa4e8fcfa None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:10.968279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ce1c3a1c-338b-494d-ad56-319aa4e8fcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:10.992483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c3a1c-338b-494d-ad56-319aa4e8fcfa None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:10.998399 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:10.999220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:10.999760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:11.000149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] PATH_INFO: `/v3/projects/bb364b49673642459ef4a069df42a157` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:11.001541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb364b49673642459ef4a069df42a157)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:11.003927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:11.094191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb364b49673642459ef4a069df42a157', 'request_id': 'req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:11.106220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb364b49673642459ef4a069df42a157', 'request_id': 'req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:11.115183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb364b49673642459ef4a069df42a157', 'request_id': 'req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:11.131304 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] The token cache is being invalidate because project bb364b49673642459ef4a069df42a157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:04:11.131892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:11.132871 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:11.134703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1164/2304] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:04:10 2022] DELETE /identity/v3/projects/bb364b49673642459ef4a069df42a157 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:11.141046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce10a37-f0ea-4133-a5c6-fb1e8f8ceaec service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:11.154428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce10a37-f0ea-4133-a5c6-fb1e8f8ceaec service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:11.199939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:11.236326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd016c-36a5-49b6-97c8-02747b44724c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:11.238681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd016c-36a5-49b6-97c8-02747b44724c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:11.238969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd016c-36a5-49b6-97c8-02747b44724c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:11.251323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd016c-36a5-49b6-97c8-02747b44724c admin admin] PATH_INFO: `/v3/users/4e90ae9eeeec404c8c0b4f46db052c50` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:11.253464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd016c-36a5-49b6-97c8-02747b44724c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e90ae9eeeec404c8c0b4f46db052c50)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:11.256492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd016c-36a5-49b6-97c8-02747b44724c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:11.341010 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3dd016c-36a5-49b6-97c8-02747b44724c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e90ae9eeeec404c8c0b4f46db052c50'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:11.359966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3dd016c-36a5-49b6-97c8-02747b44724c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e90ae9eeeec404c8c0b4f46db052c50'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:11.361087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3dd016c-36a5-49b6-97c8-02747b44724c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e90ae9eeeec404c8c0b4f46db052c50'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:11.361720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3dd016c-36a5-49b6-97c8-02747b44724c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e90ae9eeeec404c8c0b4f46db052c50'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:11.371821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3dd016c-36a5-49b6-97c8-02747b44724c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e90ae9eeeec404c8c0b4f46db052c50'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:11.390950 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1141/2305] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:04:11 2022] DELETE /identity/v3/users/4e90ae9eeeec404c8c0b4f46db052c50 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:04:12.090787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:12.095344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:12.109124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebfe0e-9095-49b0-b803-6091fd5c3b08 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:12.160855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:12.189822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:12.190590 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:12.190935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:12.191321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] PATH_INFO: `/v3/projects/252497deab5f4d4fb090687a76dcbdee` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:12.192485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=252497deab5f4d4fb090687a76dcbdee)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:12.194906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:12.246298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252497deab5f4d4fb090687a76dcbdee', 'request_id': 'req-18488924-a02c-4377-b47b-596d146b7ffb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:12.259475 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252497deab5f4d4fb090687a76dcbdee', 'request_id': 'req-18488924-a02c-4377-b47b-596d146b7ffb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:12.267817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252497deab5f4d4fb090687a76dcbdee', 'request_id': 'req-18488924-a02c-4377-b47b-596d146b7ffb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:12.280665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] The token cache is being invalidate because project 252497deab5f4d4fb090687a76dcbdee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:04:12.280953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:12.281613 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-18488924-a02c-4377-b47b-596d146b7ffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:12.283386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1165/2306] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:04:12 2022] DELETE /identity/v3/projects/252497deab5f4d4fb090687a76dcbdee => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:12.288483 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b7165-a1b9-4ed1-863f-2abb9299b2c4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:12.289406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b7165-a1b9-4ed1-863f-2abb9299b2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:12.289907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b7165-a1b9-4ed1-863f-2abb9299b2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:12.312369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-424b7165-a1b9-4ed1-863f-2abb9299b2c4 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:12.313637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-424b7165-a1b9-4ed1-863f-2abb9299b2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:12.385458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1142/2307] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:12.393658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3165501-7b10-4dce-a8bc-fc35939af305 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:12.394055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3165501-7b10-4dce-a8bc-fc35939af305 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:12.394451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3165501-7b10-4dce-a8bc-fc35939af305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:12.421187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d3165501-7b10-4dce-a8bc-fc35939af305 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:12.422500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3165501-7b10-4dce-a8bc-fc35939af305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:12.432542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1166/2308] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:12.437323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b7165-a1b9-4ed1-863f-2abb9299b2c4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:12.449018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b7165-a1b9-4ed1-863f-2abb9299b2c4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:12.454905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1803-9dac-44c8-af8a-7a67ba4d821f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:12.455546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1803-9dac-44c8-af8a-7a67ba4d821f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:12.455821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1803-9dac-44c8-af8a-7a67ba4d821f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:12.456119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1803-9dac-44c8-af8a-7a67ba4d821f admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:12.456738 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d1803-9dac-44c8-af8a-7a67ba4d821f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:04:12.457070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d1803-9dac-44c8-af8a-7a67ba4d821f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:12.458894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d1803-9dac-44c8-af8a-7a67ba4d821f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:12.474830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1143/2309] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:04:12.479249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3165501-7b10-4dce-a8bc-fc35939af305 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:12.489340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3165501-7b10-4dce-a8bc-fc35939af305 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:12.494315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fa700-a3fe-4229-8868-425b2960a66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:12.495046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fa700-a3fe-4229-8868-425b2960a66a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:12.495394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fa700-a3fe-4229-8868-425b2960a66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:12.495735 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fa700-a3fe-4229-8868-425b2960a66a admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:12.496440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fa700-a3fe-4229-8868-425b2960a66a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:12.498561 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fa700-a3fe-4229-8868-425b2960a66a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:12.532710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1167/2310] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:04:12 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:04:12.536936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1803-9dac-44c8-af8a-7a67ba4d821f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:12.548681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1803-9dac-44c8-af8a-7a67ba4d821f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:12.605355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:12.629413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d79e9-3cdb-4d29-a008-31f2159f59fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:12.630030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d79e9-3cdb-4d29-a008-31f2159f59fb admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:12.630310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d79e9-3cdb-4d29-a008-31f2159f59fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:12.630584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d79e9-3cdb-4d29-a008-31f2159f59fb admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:12.631259 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d79e9-3cdb-4d29-a008-31f2159f59fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:12.633186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d79e9-3cdb-4d29-a008-31f2159f59fb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:12.662265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1144/2311] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:04:12 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:04:12.663477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fa700-a3fe-4229-8868-425b2960a66a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:12.691607 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fa700-a3fe-4229-8868-425b2960a66a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:12.697404 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d0900-33fc-4d38-af09-8fe8e9336fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:12.698024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d0900-33fc-4d38-af09-8fe8e9336fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:12.698344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d0900-33fc-4d38-af09-8fe8e9336fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:12.698664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d0900-33fc-4d38-af09-8fe8e9336fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:12.699622 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d0900-33fc-4d38-af09-8fe8e9336fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:12.701920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d0900-33fc-4d38-af09-8fe8e9336fc1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:12.714112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1168/2312] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:04:12.719102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d79e9-3cdb-4d29-a008-31f2159f59fb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:12.732506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d79e9-3cdb-4d29-a008-31f2159f59fb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:12.738047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ae6a8-2bb2-4445-aeaa-2140093c9dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:12.740253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ae6a8-2bb2-4445-aeaa-2140093c9dfa admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:12.740874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ae6a8-2bb2-4445-aeaa-2140093c9dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:12.741582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ae6a8-2bb2-4445-aeaa-2140093c9dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:12.742582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ae6a8-2bb2-4445-aeaa-2140093c9dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:12.745825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ae6a8-2bb2-4445-aeaa-2140093c9dfa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:12.759724 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1145/2313] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:12.763967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d0900-33fc-4d38-af09-8fe8e9336fc1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:12.775751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d0900-33fc-4d38-af09-8fe8e9336fc1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:12.781440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17cc28-e7ad-4120-8ea1-73a8ff9c2a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:12.782212 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17cc28-e7ad-4120-8ea1-73a8ff9c2a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:12.782515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17cc28-e7ad-4120-8ea1-73a8ff9c2a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:12.782789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17cc28-e7ad-4120-8ea1-73a8ff9c2a95 admin admin] PATH_INFO: `/v3/projects/9f22cc224a9a4468a67757a8626eb0c0/users/4fd4dec9c3204d4eb074e7935c109b8b/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:12.799923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17cc28-e7ad-4120-8ea1-73a8ff9c2a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f22cc224a9a4468a67757a8626eb0c0, user_id=4fd4dec9c3204d4eb074e7935c109b8b, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:12.802157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17cc28-e7ad-4120-8ea1-73a8ff9c2a95 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:12.818926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1169/2314] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:12 2022] PUT /identity/v3/projects/9f22cc224a9a4468a67757a8626eb0c0/users/4fd4dec9c3204d4eb074e7935c109b8b/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:12.833643 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5ffb0-68ce-4ca5-a76a-a6d14afcd2c9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:12.833936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5ffb0-68ce-4ca5-a76a-a6d14afcd2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:12.834211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5ffb0-68ce-4ca5-a76a-a6d14afcd2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:12.865218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-06f5ffb0-68ce-4ca5-a76a-a6d14afcd2c9 None None] MFA Rules not processed for user `4fd4dec9c3204d4eb074e7935c109b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:12.866460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-06f5ffb0-68ce-4ca5-a76a-a6d14afcd2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:12.947260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1146/2315] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:12.953001 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17cc28-e7ad-4120-8ea1-73a8ff9c2a95 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:13.048861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17cc28-e7ad-4120-8ea1-73a8ff9c2a95 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:13.054088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b6fa8-888b-45dc-8e9e-50fde07b1b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:13.054698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b6fa8-888b-45dc-8e9e-50fde07b1b3f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:13.054982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b6fa8-888b-45dc-8e9e-50fde07b1b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:13.055293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b6fa8-888b-45dc-8e9e-50fde07b1b3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:13.055922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b6fa8-888b-45dc-8e9e-50fde07b1b3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:13.057815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b6fa8-888b-45dc-8e9e-50fde07b1b3f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:13.064347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:13.109705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1170/2316] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:04:12 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:04:13.112293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5ffb0-68ce-4ca5-a76a-a6d14afcd2c9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:13.134026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5ffb0-68ce-4ca5-a76a-a6d14afcd2c9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:13.246954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee56ed-2100-4f85-801a-9a5ffe5b2be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:13.248645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee56ed-2100-4f85-801a-9a5ffe5b2be0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:13.249073 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee56ed-2100-4f85-801a-9a5ffe5b2be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:13.250103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee56ed-2100-4f85-801a-9a5ffe5b2be0 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:13.251059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee56ed-2100-4f85-801a-9a5ffe5b2be0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:13.253526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee56ed-2100-4f85-801a-9a5ffe5b2be0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:13.292597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1147/2317] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:04:13 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:04:13.297245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b6fa8-888b-45dc-8e9e-50fde07b1b3f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:13.315981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b6fa8-888b-45dc-8e9e-50fde07b1b3f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:13.321422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc914a-15d4-4fff-96b1-54c9f12ef503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:13.322028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc914a-15d4-4fff-96b1-54c9f12ef503 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:13.322306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc914a-15d4-4fff-96b1-54c9f12ef503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:13.322583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc914a-15d4-4fff-96b1-54c9f12ef503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:13.323256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc914a-15d4-4fff-96b1-54c9f12ef503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:13.324986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc914a-15d4-4fff-96b1-54c9f12ef503 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:13.351313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1171/2318] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:13.353541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee56ed-2100-4f85-801a-9a5ffe5b2be0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:13.364846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee56ed-2100-4f85-801a-9a5ffe5b2be0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:13.370608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeaeccc-7527-4dea-80d7-12da733b3628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:13.371318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaeccc-7527-4dea-80d7-12da733b3628 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:13.371593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaeccc-7527-4dea-80d7-12da733b3628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:13.371870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaeccc-7527-4dea-80d7-12da733b3628 admin admin] PATH_INFO: `/v3/projects/194c101dd1eb45bc8c5d9d36b64c7382/users/6c3736ef0ca145bc966544e451d85ee8/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:13.391541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeaeccc-7527-4dea-80d7-12da733b3628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=194c101dd1eb45bc8c5d9d36b64c7382, user_id=6c3736ef0ca145bc966544e451d85ee8, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:13.395424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeaeccc-7527-4dea-80d7-12da733b3628 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:13.441271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1148/2319] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:13 2022] PUT /identity/v3/projects/194c101dd1eb45bc8c5d9d36b64c7382/users/6c3736ef0ca145bc966544e451d85ee8/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:13.446069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc914a-15d4-4fff-96b1-54c9f12ef503 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:13.457021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc914a-15d4-4fff-96b1-54c9f12ef503 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:13.529315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be943b03-45f3-4e12-80d7-343ad5ef7f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:13.529968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943b03-45f3-4e12-80d7-343ad5ef7f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:13.530249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943b03-45f3-4e12-80d7-343ad5ef7f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:13.530519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943b03-45f3-4e12-80d7-343ad5ef7f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:13.531190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be943b03-45f3-4e12-80d7-343ad5ef7f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:13.532795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be943b03-45f3-4e12-80d7-343ad5ef7f8d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:13.543946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1172/2320] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:04:13.548543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeaeccc-7527-4dea-80d7-12da733b3628 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:13.559472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeaeccc-7527-4dea-80d7-12da733b3628 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:13.565437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a0dd6-6afd-4fbf-9f3b-1220207f3503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:13.566207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a0dd6-6afd-4fbf-9f3b-1220207f3503 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:13.566608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a0dd6-6afd-4fbf-9f3b-1220207f3503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:13.567022 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a0dd6-6afd-4fbf-9f3b-1220207f3503 admin admin] PATH_INFO: `/v3/projects/194c101dd1eb45bc8c5d9d36b64c7382/users/6c3736ef0ca145bc966544e451d85ee8/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:13.569337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a0dd6-6afd-4fbf-9f3b-1220207f3503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=194c101dd1eb45bc8c5d9d36b64c7382, user_id=6c3736ef0ca145bc966544e451d85ee8, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:13.572143 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a0dd6-6afd-4fbf-9f3b-1220207f3503 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:13.585478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1149/2321] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:13 2022] PUT /identity/v3/projects/194c101dd1eb45bc8c5d9d36b64c7382/users/6c3736ef0ca145bc966544e451d85ee8/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:13.596521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13592578-9891-492e-800e-371121938e5a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:13.596884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13592578-9891-492e-800e-371121938e5a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:13.597535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13592578-9891-492e-800e-371121938e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:13.633215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-13592578-9891-492e-800e-371121938e5a None None] MFA Rules not processed for user `6c3736ef0ca145bc966544e451d85ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:13.634447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-13592578-9891-492e-800e-371121938e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:13.711532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1173/2322] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:13.723867 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a0dd6-6afd-4fbf-9f3b-1220207f3503 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:13.725990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-681a0dd6-6afd-4fbf-9f3b-1220207f3503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:13.733431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:13.816562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a0dd6-6afd-4fbf-9f3b-1220207f3503 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:13.821365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7025fd9-4de7-4d70-b1b7-90d8b6951df0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:13.823408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7025fd9-4de7-4d70-b1b7-90d8b6951df0 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:13.823768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7025fd9-4de7-4d70-b1b7-90d8b6951df0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:13.824121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7025fd9-4de7-4d70-b1b7-90d8b6951df0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:13.854349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7025fd9-4de7-4d70-b1b7-90d8b6951df0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:13.856286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7025fd9-4de7-4d70-b1b7-90d8b6951df0 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:13.880179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1150/2323] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:14.390082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13592578-9891-492e-800e-371121938e5a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:14.507896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:14.513766 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13592578-9891-492e-800e-371121938e5a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:14.523478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:14.524437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:14.527104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:14.527104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] PATH_INFO: `/v3/users/4fd4dec9c3204d4eb074e7935c109b8b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:14.527741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fd4dec9c3204d4eb074e7935c109b8b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:14.529791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:14.604629 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fd4dec9c3204d4eb074e7935c109b8b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:14.604852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fd4dec9c3204d4eb074e7935c109b8b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:14.618449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fd4dec9c3204d4eb074e7935c109b8b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:14.628371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fd4dec9c3204d4eb074e7935c109b8b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:14.645834 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fd4dec9c3204d4eb074e7935c109b8b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:14.655746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1174/2324] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:04:14 2022] DELETE /identity/v3/users/4fd4dec9c3204d4eb074e7935c109b8b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:14.660059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ed721-d542-441f-8353-99492a051254 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:14.660558 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ed721-d542-441f-8353-99492a051254 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:14.660661 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ed721-d542-441f-8353-99492a051254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:14.687892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-014ed721-d542-441f-8353-99492a051254 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:14.690179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-014ed721-d542-441f-8353-99492a051254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:14.764339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1151/2325] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:14.775893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:14.777830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:14.851186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237591aa-af44-417f-a061-cb444db530e0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:14.854767 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecf597-9725-4656-8a21-d59cf6378de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:14.855431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecf597-9725-4656-8a21-d59cf6378de8 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:14.865953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecf597-9725-4656-8a21-d59cf6378de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:14.865953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecf597-9725-4656-8a21-d59cf6378de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:14.875630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecf597-9725-4656-8a21-d59cf6378de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:14.877090 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecf597-9725-4656-8a21-d59cf6378de8 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:14.897590 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1175/2326] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:15.766068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ed721-d542-441f-8353-99492a051254 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:15.768274 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-014ed721-d542-441f-8353-99492a051254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:15.822352 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ed721-d542-441f-8353-99492a051254 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:15.837926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:15.840383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:15.842137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:15.846846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] PATH_INFO: `/v3/projects/9f22cc224a9a4468a67757a8626eb0c0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:15.851414 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f22cc224a9a4468a67757a8626eb0c0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:15.854367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:15.967134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f22cc224a9a4468a67757a8626eb0c0', 'request_id': 'req-5e9dfc15-db8c-41a3-9c1d-d1b705144403'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:15.975751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f22cc224a9a4468a67757a8626eb0c0', 'request_id': 'req-5e9dfc15-db8c-41a3-9c1d-d1b705144403'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:15.982974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f22cc224a9a4468a67757a8626eb0c0', 'request_id': 'req-5e9dfc15-db8c-41a3-9c1d-d1b705144403'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:16.002094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] The token cache is being invalidate because project 9f22cc224a9a4468a67757a8626eb0c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:04:16.002665 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:16.003570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:16.005279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1152/2327] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:04:15 2022] DELETE /identity/v3/projects/9f22cc224a9a4468a67757a8626eb0c0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:16.009316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecf597-9725-4656-8a21-d59cf6378de8 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:16.024308 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecf597-9725-4656-8a21-d59cf6378de8 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:16.120897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508be080-795b-4e54-a034-21edc708dcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:16.121817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be080-795b-4e54-a034-21edc708dcfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:16.122875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be080-795b-4e54-a034-21edc708dcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:16.125364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be080-795b-4e54-a034-21edc708dcfa admin admin] PATH_INFO: `/v3/users/6c3736ef0ca145bc966544e451d85ee8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:16.127323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508be080-795b-4e54-a034-21edc708dcfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c3736ef0ca145bc966544e451d85ee8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:16.129352 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508be080-795b-4e54-a034-21edc708dcfa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:16.160916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:16.214150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-508be080-795b-4e54-a034-21edc708dcfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c3736ef0ca145bc966544e451d85ee8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:16.215110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-508be080-795b-4e54-a034-21edc708dcfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c3736ef0ca145bc966544e451d85ee8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:16.230747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-508be080-795b-4e54-a034-21edc708dcfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c3736ef0ca145bc966544e451d85ee8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:16.245187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-508be080-795b-4e54-a034-21edc708dcfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c3736ef0ca145bc966544e451d85ee8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:16.261528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-508be080-795b-4e54-a034-21edc708dcfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c3736ef0ca145bc966544e451d85ee8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:16.289996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1176/2328] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:04:16 2022] DELETE /identity/v3/users/6c3736ef0ca145bc966544e451d85ee8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:16.865156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:16.869748 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:16.886904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dfc15-db8c-41a3-9c1d-d1b705144403 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:16.938253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:17.029424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:17.030091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:17.030377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:17.030652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] PATH_INFO: `/v3/projects/194c101dd1eb45bc8c5d9d36b64c7382` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:17.032563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=194c101dd1eb45bc8c5d9d36b64c7382)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:17.034716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:17.136232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194c101dd1eb45bc8c5d9d36b64c7382', 'request_id': 'req-e3f3da8b-3e20-4722-b533-7bde0f36c260'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:17.156400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194c101dd1eb45bc8c5d9d36b64c7382', 'request_id': 'req-e3f3da8b-3e20-4722-b533-7bde0f36c260'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:17.168233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194c101dd1eb45bc8c5d9d36b64c7382', 'request_id': 'req-e3f3da8b-3e20-4722-b533-7bde0f36c260'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:17.187544 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] The token cache is being invalidate because project 194c101dd1eb45bc8c5d9d36b64c7382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:04:17.187861 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:17.188590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3f3da8b-3e20-4722-b533-7bde0f36c260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:17.193914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1153/2329] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:04:16 2022] DELETE /identity/v3/projects/194c101dd1eb45bc8c5d9d36b64c7382 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:04:17.195872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad0030-841b-4513-a1b5-702bab18fd2b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:17.196373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad0030-841b-4513-a1b5-702bab18fd2b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:17.196620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad0030-841b-4513-a1b5-702bab18fd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:17.244489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fdad0030-841b-4513-a1b5-702bab18fd2b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:17.245820 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fdad0030-841b-4513-a1b5-702bab18fd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:17.351291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1177/2330] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:17.362579 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6e180-24e4-4cf6-b0e6-4f10aa35534a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:17.362944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6e180-24e4-4cf6-b0e6-4f10aa35534a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:17.363169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6e180-24e4-4cf6-b0e6-4f10aa35534a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:17.384639 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e8a6e180-24e4-4cf6-b0e6-4f10aa35534a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:17.385503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e8a6e180-24e4-4cf6-b0e6-4f10aa35534a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:17.401514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad0030-841b-4513-a1b5-702bab18fd2b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:17.411315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1154/2331] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:17.416827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad0030-841b-4513-a1b5-702bab18fd2b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:17.424851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bacecb-4eb7-4604-ba44-a904f60a3338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:17.425646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bacecb-4eb7-4604-ba44-a904f60a3338 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:17.425923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bacecb-4eb7-4604-ba44-a904f60a3338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:17.426196 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bacecb-4eb7-4604-ba44-a904f60a3338 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:17.426844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bacecb-4eb7-4604-ba44-a904f60a3338 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:04:17.427176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bacecb-4eb7-4604-ba44-a904f60a3338 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:17.428767 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bacecb-4eb7-4604-ba44-a904f60a3338 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:17.453889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1178/2332] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:04:17.458244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6e180-24e4-4cf6-b0e6-4f10aa35534a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:17.482700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6e180-24e4-4cf6-b0e6-4f10aa35534a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:17.490633 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0d45f-bca5-4778-9ec8-6b196fc30f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:17.493457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0d45f-bca5-4778-9ec8-6b196fc30f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:17.493832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0d45f-bca5-4778-9ec8-6b196fc30f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:17.494167 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0d45f-bca5-4778-9ec8-6b196fc30f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:17.495025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0d45f-bca5-4778-9ec8-6b196fc30f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:17.498007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0d45f-bca5-4778-9ec8-6b196fc30f9d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:17.553112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bacecb-4eb7-4604-ba44-a904f60a3338 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:17.560842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1155/2333] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:04:17 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:04:17.565723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bacecb-4eb7-4604-ba44-a904f60a3338 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:17.670148 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fac383-d701-45b7-a98d-ebb275eb35d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:17.670583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac383-d701-45b7-a98d-ebb275eb35d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:17.670866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac383-d701-45b7-a98d-ebb275eb35d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:17.671180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac383-d701-45b7-a98d-ebb275eb35d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:17.671816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fac383-d701-45b7-a98d-ebb275eb35d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:17.674274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fac383-d701-45b7-a98d-ebb275eb35d4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:17.730547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1179/2334] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:04:17 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:04:17.732949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0d45f-bca5-4778-9ec8-6b196fc30f9d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:17.748097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0d45f-bca5-4778-9ec8-6b196fc30f9d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:17.762627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c1f79-965f-4dcb-bd1f-3fd8a954c595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:17.763620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c1f79-965f-4dcb-bd1f-3fd8a954c595 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:17.763989 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c1f79-965f-4dcb-bd1f-3fd8a954c595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:17.764395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c1f79-965f-4dcb-bd1f-3fd8a954c595 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:17.765240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c1f79-965f-4dcb-bd1f-3fd8a954c595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:17.769242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c1f79-965f-4dcb-bd1f-3fd8a954c595 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:17.790580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1156/2335] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:17.794976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fac383-d701-45b7-a98d-ebb275eb35d4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:17.821975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fac383-d701-45b7-a98d-ebb275eb35d4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:17.834303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07260f3f-0050-4f53-bea3-e4d723987144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:17.835439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260f3f-0050-4f53-bea3-e4d723987144 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:17.836137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260f3f-0050-4f53-bea3-e4d723987144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:17.836610 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260f3f-0050-4f53-bea3-e4d723987144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:17.837504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07260f3f-0050-4f53-bea3-e4d723987144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:17.842708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07260f3f-0050-4f53-bea3-e4d723987144 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:17.867905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1180/2336] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:17.870129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c1f79-965f-4dcb-bd1f-3fd8a954c595 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:17.891186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c1f79-965f-4dcb-bd1f-3fd8a954c595 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:17.905881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c3f8c-f81b-4006-8614-6811853a07f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:17.906741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c3f8c-f81b-4006-8614-6811853a07f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:17.907400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c3f8c-f81b-4006-8614-6811853a07f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:17.907400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c3f8c-f81b-4006-8614-6811853a07f4 admin admin] PATH_INFO: `/v3/projects/7264402a31974c92970a0e22692c6cc4/users/446d5584d65d41a393e35e8cd3943781/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:17.927770 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c3f8c-f81b-4006-8614-6811853a07f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7264402a31974c92970a0e22692c6cc4, user_id=446d5584d65d41a393e35e8cd3943781, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:17.931180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c3f8c-f81b-4006-8614-6811853a07f4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:17.951421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1157/2337] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:17 2022] PUT /identity/v3/projects/7264402a31974c92970a0e22692c6cc4/users/446d5584d65d41a393e35e8cd3943781/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:17.964644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d6163-47d6-4bfc-a908-35fe9f77f8ec None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:17.965246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d6163-47d6-4bfc-a908-35fe9f77f8ec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:17.965692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d6163-47d6-4bfc-a908-35fe9f77f8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:17.980330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:18.030323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b18d6163-47d6-4bfc-a908-35fe9f77f8ec None None] MFA Rules not processed for user `446d5584d65d41a393e35e8cd3943781`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:18.032929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b18d6163-47d6-4bfc-a908-35fe9f77f8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:18.204139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1181/2338] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:18.218973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c3f8c-f81b-4006-8614-6811853a07f4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:18.222709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d13c3f8c-f81b-4006-8614-6811853a07f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:18.340676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c3f8c-f81b-4006-8614-6811853a07f4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:18.345953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471fdf1-ad6f-466d-91a8-11f23de73a0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:18.346529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471fdf1-ad6f-466d-91a8-11f23de73a0d service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:18.347145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471fdf1-ad6f-466d-91a8-11f23de73a0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:18.347287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471fdf1-ad6f-466d-91a8-11f23de73a0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:18.357704 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471fdf1-ad6f-466d-91a8-11f23de73a0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:18.360666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471fdf1-ad6f-466d-91a8-11f23de73a0d service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:18.371590 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:18.383956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1158/2339] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:04:28.042899 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81458c33-307d-4e50-bcb4-6207144bed6f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:28.043437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81458c33-307d-4e50-bcb4-6207144bed6f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:28.043535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81458c33-307d-4e50-bcb4-6207144bed6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:28.062493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-81458c33-307d-4e50-bcb4-6207144bed6f None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:28.063720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-81458c33-307d-4e50-bcb4-6207144bed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:28.118633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1182/2340] 10.222.0.86 () {58 vars in 967 bytes} [Mon Aug 29 19:04:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:28.131838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471fdf1-ad6f-466d-91a8-11f23de73a0d service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:28.133250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1471fdf1-ad6f-466d-91a8-11f23de73a0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:28.202565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471fdf1-ad6f-466d-91a8-11f23de73a0d service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:28.207476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68a22e-aff0-4096-ae0b-3daf5eb4e925 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:28.208075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68a22e-aff0-4096-ae0b-3daf5eb4e925 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:28.208351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68a22e-aff0-4096-ae0b-3daf5eb4e925 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:28.208619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68a22e-aff0-4096-ae0b-3daf5eb4e925 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:28.217081 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68a22e-aff0-4096-ae0b-3daf5eb4e925 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:28.218648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68a22e-aff0-4096-ae0b-3daf5eb4e925 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:28.226479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1159/2341] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:04:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:04:34.456763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81458c33-307d-4e50-bcb4-6207144bed6f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:34.458444 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-81458c33-307d-4e50-bcb4-6207144bed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:34.525129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81458c33-307d-4e50-bcb4-6207144bed6f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:34.529794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08913e23-4655-4693-9b40-59113179ba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:34.530519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08913e23-4655-4693-9b40-59113179ba15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:34.531013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08913e23-4655-4693-9b40-59113179ba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:34.531337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08913e23-4655-4693-9b40-59113179ba15 admin admin] PATH_INFO: `/v3/users/41227378a57c43219e4ae460c642f834` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:34.533189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08913e23-4655-4693-9b40-59113179ba15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41227378a57c43219e4ae460c642f834)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:34.535810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08913e23-4655-4693-9b40-59113179ba15 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:34.577425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:34.583125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-08913e23-4655-4693-9b40-59113179ba15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41227378a57c43219e4ae460c642f834'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:34.583902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-08913e23-4655-4693-9b40-59113179ba15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41227378a57c43219e4ae460c642f834'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:34.595834 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-08913e23-4655-4693-9b40-59113179ba15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41227378a57c43219e4ae460c642f834'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:34.603202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-08913e23-4655-4693-9b40-59113179ba15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41227378a57c43219e4ae460c642f834'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:34.610569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-08913e23-4655-4693-9b40-59113179ba15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41227378a57c43219e4ae460c642f834'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:34.616339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1183/2342] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:04:34 2022] DELETE /identity/v3/users/41227378a57c43219e4ae460c642f834 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:34.801568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68a22e-aff0-4096-ae0b-3daf5eb4e925 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:34.825982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68a22e-aff0-4096-ae0b-3daf5eb4e925 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:34.885238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:34.885872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:34.886157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:34.886436 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] PATH_INFO: `/v3/projects/cbf40ccacdc74575909f541214adf6f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:34.887625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf40ccacdc74575909f541214adf6f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:34.889545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:34.939606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf40ccacdc74575909f541214adf6f7', 'request_id': 'req-402bbc46-b170-4825-85ae-f753cc6715ae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:34.946067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf40ccacdc74575909f541214adf6f7', 'request_id': 'req-402bbc46-b170-4825-85ae-f753cc6715ae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:34.952129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf40ccacdc74575909f541214adf6f7', 'request_id': 'req-402bbc46-b170-4825-85ae-f753cc6715ae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:34.964447 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] The token cache is being invalidate because project cbf40ccacdc74575909f541214adf6f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:04:34.964654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:34.965298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-402bbc46-b170-4825-85ae-f753cc6715ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:34.967852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1160/2343] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:04:34 2022] DELETE /identity/v3/projects/cbf40ccacdc74575909f541214adf6f7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:34.974005 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e97f1-65a5-447e-8f2b-a9b16c85d59e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:34.974299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e97f1-65a5-447e-8f2b-a9b16c85d59e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:34.974595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e97f1-65a5-447e-8f2b-a9b16c85d59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:35.001610 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ca4e97f1-65a5-447e-8f2b-a9b16c85d59e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:35.003177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ca4e97f1-65a5-447e-8f2b-a9b16c85d59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:35.107155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1184/2344] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:35.115555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53de1e-e96e-4207-892d-98ee2890b222 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:35.115835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53de1e-e96e-4207-892d-98ee2890b222 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:35.116122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53de1e-e96e-4207-892d-98ee2890b222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:35.140026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6f53de1e-e96e-4207-892d-98ee2890b222 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:35.142160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6f53de1e-e96e-4207-892d-98ee2890b222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:35.152083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1161/2345] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:35.157215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e97f1-65a5-447e-8f2b-a9b16c85d59e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:35.167447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e97f1-65a5-447e-8f2b-a9b16c85d59e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:35.172929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d5cb7-5bf6-42f3-93a1-35234bf25cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:35.173533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d5cb7-5bf6-42f3-93a1-35234bf25cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:35.173819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d5cb7-5bf6-42f3-93a1-35234bf25cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:35.174085 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d5cb7-5bf6-42f3-93a1-35234bf25cbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:35.174731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d5cb7-5bf6-42f3-93a1-35234bf25cbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:04:35.175057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d5cb7-5bf6-42f3-93a1-35234bf25cbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:35.176754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d5cb7-5bf6-42f3-93a1-35234bf25cbd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:35.191072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1185/2346] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:04:35.194375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53de1e-e96e-4207-892d-98ee2890b222 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:35.204363 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53de1e-e96e-4207-892d-98ee2890b222 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:35.209180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a9de0-9535-411b-bf82-fad04b385897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:35.209786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a9de0-9535-411b-bf82-fad04b385897 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:35.210067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a9de0-9535-411b-bf82-fad04b385897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:35.210337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a9de0-9535-411b-bf82-fad04b385897 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:35.210959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a9de0-9535-411b-bf82-fad04b385897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:35.212925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a9de0-9535-411b-bf82-fad04b385897 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:35.225239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:35.249721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1162/2347] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:04:35 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:04:35.253843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d5cb7-5bf6-42f3-93a1-35234bf25cbd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:35.265464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d5cb7-5bf6-42f3-93a1-35234bf25cbd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:35.357208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28655ec-c58d-48bb-8983-5a8957e6ca4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:35.357992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28655ec-c58d-48bb-8983-5a8957e6ca4f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:35.358298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28655ec-c58d-48bb-8983-5a8957e6ca4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:35.358582 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28655ec-c58d-48bb-8983-5a8957e6ca4f admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:35.359462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28655ec-c58d-48bb-8983-5a8957e6ca4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:35.361633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28655ec-c58d-48bb-8983-5a8957e6ca4f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:35.399041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1186/2348] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:04:35 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:04:35.400404 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a9de0-9535-411b-bf82-fad04b385897 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:35.416417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a9de0-9535-411b-bf82-fad04b385897 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:35.432008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a144d56-cddf-4f31-bfb5-8733fa07cf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:35.432855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144d56-cddf-4f31-bfb5-8733fa07cf46 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:35.433301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144d56-cddf-4f31-bfb5-8733fa07cf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:35.433637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144d56-cddf-4f31-bfb5-8733fa07cf46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:35.438641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a144d56-cddf-4f31-bfb5-8733fa07cf46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:35.440904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a144d56-cddf-4f31-bfb5-8733fa07cf46 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:35.466868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28655ec-c58d-48bb-8983-5a8957e6ca4f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:35.469364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1163/2349] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:35.479896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28655ec-c58d-48bb-8983-5a8957e6ca4f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:35.487522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a7db1-b23e-4607-89dd-0b1076c5e3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:35.488932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a7db1-b23e-4607-89dd-0b1076c5e3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:35.489394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a7db1-b23e-4607-89dd-0b1076c5e3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:35.489824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a7db1-b23e-4607-89dd-0b1076c5e3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:35.491013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a7db1-b23e-4607-89dd-0b1076c5e3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:35.493422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a7db1-b23e-4607-89dd-0b1076c5e3e5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:35.510028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1187/2350] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:04:35.514720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a144d56-cddf-4f31-bfb5-8733fa07cf46 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:35.525452 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a144d56-cddf-4f31-bfb5-8733fa07cf46 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:35.531410 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f42cb6-8fa0-427b-aea8-c31ad8dcfd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:35.532846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f42cb6-8fa0-427b-aea8-c31ad8dcfd64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:35.532846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f42cb6-8fa0-427b-aea8-c31ad8dcfd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:35.532846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f42cb6-8fa0-427b-aea8-c31ad8dcfd64 admin admin] PATH_INFO: `/v3/projects/7918fb7f8aea4f58857307485bc09ec6/users/98e853439341466dbea3a535ca13d12b/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:35.548787 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f42cb6-8fa0-427b-aea8-c31ad8dcfd64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7918fb7f8aea4f58857307485bc09ec6, user_id=98e853439341466dbea3a535ca13d12b, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:35.551846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f42cb6-8fa0-427b-aea8-c31ad8dcfd64 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:35.573275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1164/2351] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:35 2022] PUT /identity/v3/projects/7918fb7f8aea4f58857307485bc09ec6/users/98e853439341466dbea3a535ca13d12b/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:35.576213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c1c8-bc85-446c-ad14-dc3cd1d1be81 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:35.576504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c1c8-bc85-446c-ad14-dc3cd1d1be81 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:35.576774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c1c8-bc85-446c-ad14-dc3cd1d1be81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:35.598626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c015c1c8-bc85-446c-ad14-dc3cd1d1be81 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:35.600417 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c015c1c8-bc85-446c-ad14-dc3cd1d1be81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:35.625301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:35.667566 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1188/2352] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:35.677117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f42cb6-8fa0-427b-aea8-c31ad8dcfd64 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:35.678809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-26f42cb6-8fa0-427b-aea8-c31ad8dcfd64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:35.735821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f42cb6-8fa0-427b-aea8-c31ad8dcfd64 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:35.739305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099241c-5cbb-46f7-a4d7-37c30edaa173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:35.739900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099241c-5cbb-46f7-a4d7-37c30edaa173 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:35.740178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099241c-5cbb-46f7-a4d7-37c30edaa173 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:35.740450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099241c-5cbb-46f7-a4d7-37c30edaa173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:35.749679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099241c-5cbb-46f7-a4d7-37c30edaa173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:35.751112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099241c-5cbb-46f7-a4d7-37c30edaa173 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:35.760318 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1165/2353] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:36.531687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3b2a3-4727-4c9d-85b1-45917acfaada None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:36.532080 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3b2a3-4727-4c9d-85b1-45917acfaada None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:36.532242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3b2a3-4727-4c9d-85b1-45917acfaada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:36.567484 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-07a3b2a3-4727-4c9d-85b1-45917acfaada None None] MFA Rules not processed for user `98e853439341466dbea3a535ca13d12b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:36.569144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-07a3b2a3-4727-4c9d-85b1-45917acfaada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:36.652779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1189/2354] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:36.664213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099241c-5cbb-46f7-a4d7-37c30edaa173 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:36.666205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1099241c-5cbb-46f7-a4d7-37c30edaa173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:36.770120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099241c-5cbb-46f7-a4d7-37c30edaa173 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:36.773699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08af99-d4b8-48e4-a502-f2e9f91a2876 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:36.774298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08af99-d4b8-48e4-a502-f2e9f91a2876 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:36.774574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08af99-d4b8-48e4-a502-f2e9f91a2876 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:36.774842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08af99-d4b8-48e4-a502-f2e9f91a2876 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:36.785132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08af99-d4b8-48e4-a502-f2e9f91a2876 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:36.786553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08af99-d4b8-48e4-a502-f2e9f91a2876 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:36.795763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1166/2355] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:42.141588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3b2a3-4727-4c9d-85b1-45917acfaada None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:42.143098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-07a3b2a3-4727-4c9d-85b1-45917acfaada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:42.244043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3b2a3-4727-4c9d-85b1-45917acfaada None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:42.255767 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4b100-bfc7-47bd-8c86-e7cd88f03454 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:42.256672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4b100-bfc7-47bd-8c86-e7cd88f03454 service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:42.257151 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4b100-bfc7-47bd-8c86-e7cd88f03454 service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:42.257617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4b100-bfc7-47bd-8c86-e7cd88f03454 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:42.259784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ebb4b100-bfc7-47bd-8c86-e7cd88f03454 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:42.273639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4b100-bfc7-47bd-8c86-e7cd88f03454 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:42.275349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4b100-bfc7-47bd-8c86-e7cd88f03454 service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:42.286722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1190/2356] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:43.437903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a504ee-9955-4e22-a036-d103f2cda92d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:43.438366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a504ee-9955-4e22-a036-d103f2cda92d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:43.438457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a504ee-9955-4e22-a036-d103f2cda92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:43.460929 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-75a504ee-9955-4e22-a036-d103f2cda92d None None] MFA Rules not processed for user `98e853439341466dbea3a535ca13d12b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:43.462154 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-75a504ee-9955-4e22-a036-d103f2cda92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:43.469884 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1167/2357] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:45.345075 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b9a8b-300a-4070-a34f-8af515b0991c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:45.345579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b9a8b-300a-4070-a34f-8af515b0991c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:45.345700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b9a8b-300a-4070-a34f-8af515b0991c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:45.367079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b14b9a8b-300a-4070-a34f-8af515b0991c None None] MFA Rules not processed for user `962284e9a353430881bc119e5f80a890`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:45.368600 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b14b9a8b-300a-4070-a34f-8af515b0991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:45.416564 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:45.433146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1191/2358] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:46.999688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a504ee-9955-4e22-a036-d103f2cda92d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:47.001433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-75a504ee-9955-4e22-a036-d103f2cda92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:47.017236 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a504ee-9955-4e22-a036-d103f2cda92d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:47.022039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:47.022664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:47.022948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:47.023260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] PATH_INFO: `/v3/users/446d5584d65d41a393e35e8cd3943781` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:47.024381 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=446d5584d65d41a393e35e8cd3943781)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:47.026300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:47.096216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '446d5584d65d41a393e35e8cd3943781'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:47.099553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:47.117026 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '446d5584d65d41a393e35e8cd3943781'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:47.118545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '446d5584d65d41a393e35e8cd3943781'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:47.119807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '446d5584d65d41a393e35e8cd3943781'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:47.136345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '446d5584d65d41a393e35e8cd3943781'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:47.175707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1168/2359] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:04:46 2022] DELETE /identity/v3/users/446d5584d65d41a393e35e8cd3943781 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:04:47.180176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac68f0-d57c-4d3c-8b5f-dbbbdca97b85 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:47.180719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac68f0-d57c-4d3c-8b5f-dbbbdca97b85 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:47.181783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac68f0-d57c-4d3c-8b5f-dbbbdca97b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:47.220266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5dac68f0-d57c-4d3c-8b5f-dbbbdca97b85 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:47.221935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5dac68f0-d57c-4d3c-8b5f-dbbbdca97b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:47.294776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1192/2360] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:47.304766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:47.306361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:47.371941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a638a4-fb3c-41de-b9e7-922bcd6c2f39 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:47.377932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d94291-9665-40d3-ae4e-d90177f2bc21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:47.378750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d94291-9665-40d3-ae4e-d90177f2bc21 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:47.379057 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d94291-9665-40d3-ae4e-d90177f2bc21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:47.379696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d94291-9665-40d3-ae4e-d90177f2bc21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:47.390804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d94291-9665-40d3-ae4e-d90177f2bc21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:47.392272 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d94291-9665-40d3-ae4e-d90177f2bc21 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:47.403848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1169/2361] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:47.754480 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac68f0-d57c-4d3c-8b5f-dbbbdca97b85 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:47.756137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5dac68f0-d57c-4d3c-8b5f-dbbbdca97b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:47.773029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac68f0-d57c-4d3c-8b5f-dbbbdca97b85 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:47.778047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:47.779958 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:47.780387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:47.780840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] PATH_INFO: `/v3/projects/7264402a31974c92970a0e22692c6cc4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:47.782863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7264402a31974c92970a0e22692c6cc4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:47.785156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:47.843650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7264402a31974c92970a0e22692c6cc4', 'request_id': 'req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:47.851665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7264402a31974c92970a0e22692c6cc4', 'request_id': 'req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:47.859231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7264402a31974c92970a0e22692c6cc4', 'request_id': 'req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:47.869818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] The token cache is being invalidate because project 7264402a31974c92970a0e22692c6cc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:04:47.870104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:47.870810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a6cfa16-e1f3-4a32-ba1d-5963d4cf6fdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:47.872382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1193/2362] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:04:47 2022] DELETE /identity/v3/projects/7264402a31974c92970a0e22692c6cc4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:47.877298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9e4d-f721-41d6-bec5-4299b3fe2e5c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:47.877595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9e4d-f721-41d6-bec5-4299b3fe2e5c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:47.877875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9e4d-f721-41d6-bec5-4299b3fe2e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:47.897778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b6de9e4d-f721-41d6-bec5-4299b3fe2e5c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:47.899021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b6de9e4d-f721-41d6-bec5-4299b3fe2e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:47.973130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1170/2363] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:47.980744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bc03b-c779-47a9-a016-e16b730662d2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:47.980992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bc03b-c779-47a9-a016-e16b730662d2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:47.981483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bc03b-c779-47a9-a016-e16b730662d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:48.002016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-910bc03b-c779-47a9-a016-e16b730662d2 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:48.003274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-910bc03b-c779-47a9-a016-e16b730662d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:48.011551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1194/2364] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:48.016386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de9e4d-f721-41d6-bec5-4299b3fe2e5c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:48.031223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de9e4d-f721-41d6-bec5-4299b3fe2e5c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:48.036333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e214bf-1846-4588-817d-0f468fc2788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:48.036954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e214bf-1846-4588-817d-0f468fc2788c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:48.037256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e214bf-1846-4588-817d-0f468fc2788c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:48.037507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e214bf-1846-4588-817d-0f468fc2788c admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:48.038408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e214bf-1846-4588-817d-0f468fc2788c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:04:48.038719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e214bf-1846-4588-817d-0f468fc2788c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:48.040755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e214bf-1846-4588-817d-0f468fc2788c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:48.053458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1171/2365] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:04:48.058552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bc03b-c779-47a9-a016-e16b730662d2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:48.068323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bc03b-c779-47a9-a016-e16b730662d2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:48.073298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678e385-3675-4702-bbb2-83aaabfcab4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:48.073960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678e385-3675-4702-bbb2-83aaabfcab4a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:48.074240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678e385-3675-4702-bbb2-83aaabfcab4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:48.074510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678e385-3675-4702-bbb2-83aaabfcab4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:48.075742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678e385-3675-4702-bbb2-83aaabfcab4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:48.077630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678e385-3675-4702-bbb2-83aaabfcab4a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:48.108322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1195/2366] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:04:48 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:04:48.112624 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e214bf-1846-4588-817d-0f468fc2788c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:48.125625 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e214bf-1846-4588-817d-0f468fc2788c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:48.212845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:48.219726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05263b3-08e8-4d06-b203-2e50d6f5d865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:48.220718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05263b3-08e8-4d06-b203-2e50d6f5d865 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:48.221160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05263b3-08e8-4d06-b203-2e50d6f5d865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:48.221440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05263b3-08e8-4d06-b203-2e50d6f5d865 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:48.222383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05263b3-08e8-4d06-b203-2e50d6f5d865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:48.236438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05263b3-08e8-4d06-b203-2e50d6f5d865 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:48.305806 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1172/2367] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:04:48 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:04:48.311427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678e385-3675-4702-bbb2-83aaabfcab4a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:48.332679 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678e385-3675-4702-bbb2-83aaabfcab4a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:48.359967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182bfae-a486-47ab-9bd1-a290d1857cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:48.360605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182bfae-a486-47ab-9bd1-a290d1857cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:48.361088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182bfae-a486-47ab-9bd1-a290d1857cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:48.361510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182bfae-a486-47ab-9bd1-a290d1857cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:48.362348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182bfae-a486-47ab-9bd1-a290d1857cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:48.366437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182bfae-a486-47ab-9bd1-a290d1857cd8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:48.382566 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1196/2368] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:48.386749 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05263b3-08e8-4d06-b203-2e50d6f5d865 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:48.397962 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05263b3-08e8-4d06-b203-2e50d6f5d865 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:48.408297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fe7d2-e3b9-4099-b59b-5123bb53f684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:48.408939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fe7d2-e3b9-4099-b59b-5123bb53f684 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:48.409222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fe7d2-e3b9-4099-b59b-5123bb53f684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:48.409494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fe7d2-e3b9-4099-b59b-5123bb53f684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:48.410152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fe7d2-e3b9-4099-b59b-5123bb53f684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:48.411799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fe7d2-e3b9-4099-b59b-5123bb53f684 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:48.427071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1173/2369] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:04:48.429989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182bfae-a486-47ab-9bd1-a290d1857cd8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:48.442827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182bfae-a486-47ab-9bd1-a290d1857cd8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:48.448191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e490d23-bc87-415c-b577-b809ed09949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:48.448858 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e490d23-bc87-415c-b577-b809ed09949b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:48.449762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e490d23-bc87-415c-b577-b809ed09949b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:48.449762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e490d23-bc87-415c-b577-b809ed09949b admin admin] PATH_INFO: `/v3/projects/1f7bebc3bd65445ea1f3c7a47e4dbaa2/users/2a7a359fc5974669a5dd8c59bd465b7e/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:48.465354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e490d23-bc87-415c-b577-b809ed09949b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7bebc3bd65445ea1f3c7a47e4dbaa2, user_id=2a7a359fc5974669a5dd8c59bd465b7e, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:48.468183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e490d23-bc87-415c-b577-b809ed09949b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:48.475897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a91af0-0a7c-47ca-a79b-c13155d946e8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:48.476268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a91af0-0a7c-47ca-a79b-c13155d946e8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:48.476613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a91af0-0a7c-47ca-a79b-c13155d946e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:48.496373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1197/2370] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:48 2022] PUT /identity/v3/projects/1f7bebc3bd65445ea1f3c7a47e4dbaa2/users/2a7a359fc5974669a5dd8c59bd465b7e/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:48.498954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae13d46-2292-4bc6-81d8-ad8e05276537 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:48.499273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae13d46-2292-4bc6-81d8-ad8e05276537 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:48.499548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae13d46-2292-4bc6-81d8-ad8e05276537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:48.503687 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-47a91af0-0a7c-47ca-a79b-c13155d946e8 None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:48.505314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-47a91af0-0a7c-47ca-a79b-c13155d946e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:48.521006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:48.531025 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8ae13d46-2292-4bc6-81d8-ad8e05276537 None None] MFA Rules not processed for user `2a7a359fc5974669a5dd8c59bd465b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:48.532266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8ae13d46-2292-4bc6-81d8-ad8e05276537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:48.591493 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1174/2371] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:04:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:48.608850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a91af0-0a7c-47ca-a79b-c13155d946e8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:48.611191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-47a91af0-0a7c-47ca-a79b-c13155d946e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:48.620882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1198/2372] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:48.634570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae13d46-2292-4bc6-81d8-ad8e05276537 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:48.636242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8ae13d46-2292-4bc6-81d8-ad8e05276537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:48.711078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a91af0-0a7c-47ca-a79b-c13155d946e8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:48.718809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c7a15-2b51-496b-b50c-ada8da4a7b51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:48.719481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7a15-2b51-496b-b50c-ada8da4a7b51 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:48.719764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7a15-2b51-496b-b50c-ada8da4a7b51 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:48.720043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7a15-2b51-496b-b50c-ada8da4a7b51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:48.732971 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c7a15-2b51-496b-b50c-ada8da4a7b51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:48.734567 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c7a15-2b51-496b-b50c-ada8da4a7b51 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:48.760189 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1175/2373] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:04:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 19:04:48.769722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae13d46-2292-4bc6-81d8-ad8e05276537 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:48.774212 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf73d35-55cb-40d3-9e99-c4fd238496cd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af69888b04c45b09e34d9ff7a0b564f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:48.774894 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf73d35-55cb-40d3-9e99-c4fd238496cd service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:48.775201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf73d35-55cb-40d3-9e99-c4fd238496cd service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:48.776250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf73d35-55cb-40d3-9e99-c4fd238496cd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:48.806258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf73d35-55cb-40d3-9e99-c4fd238496cd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:48.807764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf73d35-55cb-40d3-9e99-c4fd238496cd service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:48.846344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1199/2374] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:04:49.553844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995c615-8926-4776-a737-1e6fe4f46aa3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:49.554342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995c615-8926-4776-a737-1e6fe4f46aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:49.554438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995c615-8926-4776-a737-1e6fe4f46aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:49.577064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b995c615-8926-4776-a737-1e6fe4f46aa3 None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:49.578687 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b995c615-8926-4776-a737-1e6fe4f46aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:49.587405 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1176/2375] 10.222.0.86 () {58 vars in 967 bytes} [Mon Aug 29 19:04:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:49.599888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf73d35-55cb-40d3-9e99-c4fd238496cd service glance] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:49.612672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf73d35-55cb-40d3-9e99-c4fd238496cd service glance] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:49.617878 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58b9a2-4682-4eb5-9108-d9f865ae1d8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:49.618493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58b9a2-4682-4eb5-9108-d9f865ae1d8b service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:49.618774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58b9a2-4682-4eb5-9108-d9f865ae1d8b service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:49.619086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58b9a2-4682-4eb5-9108-d9f865ae1d8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:49.628555 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58b9a2-4682-4eb5-9108-d9f865ae1d8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:49.630129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58b9a2-4682-4eb5-9108-d9f865ae1d8b service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:49.638674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1200/2376] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:04:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:04:49.877291 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d88ba-5343-475f-97a0-91636e6ecebf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:49.877870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d88ba-5343-475f-97a0-91636e6ecebf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:49.878129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d88ba-5343-475f-97a0-91636e6ecebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:49.897597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3e1d88ba-5343-475f-97a0-91636e6ecebf None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:49.898895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3e1d88ba-5343-475f-97a0-91636e6ecebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:49.907621 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1177/2377] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:04:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:49.921634 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58b9a2-4682-4eb5-9108-d9f865ae1d8b service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:49.936174 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58b9a2-4682-4eb5-9108-d9f865ae1d8b service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:49.942909 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556a2f6-7978-4914-9166-66f73eaaa988 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:49.943990 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556a2f6-7978-4914-9166-66f73eaaa988 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:49.944275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556a2f6-7978-4914-9166-66f73eaaa988 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:49.944543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556a2f6-7978-4914-9166-66f73eaaa988 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:49.953961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556a2f6-7978-4914-9166-66f73eaaa988 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:49.955567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556a2f6-7978-4914-9166-66f73eaaa988 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:49.964670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1201/2378] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:04:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:04:50.309547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decef450-b218-487b-bd4a-1ab28a72a9d6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:50.309831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decef450-b218-487b-bd4a-1ab28a72a9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:50.310106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decef450-b218-487b-bd4a-1ab28a72a9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:50.330509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-decef450-b218-487b-bd4a-1ab28a72a9d6 None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:50.331807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-decef450-b218-487b-bd4a-1ab28a72a9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:50.342755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1178/2379] 10.222.0.86 () {58 vars in 967 bytes} [Mon Aug 29 19:04:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:50.355484 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556a2f6-7978-4914-9166-66f73eaaa988 service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:50.365917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556a2f6-7978-4914-9166-66f73eaaa988 service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:50.371107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f209fc3-1b8d-4434-9a54-0ffe723f2c32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:50.371797 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f209fc3-1b8d-4434-9a54-0ffe723f2c32 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:50.372105 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f209fc3-1b8d-4434-9a54-0ffe723f2c32 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:50.372371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f209fc3-1b8d-4434-9a54-0ffe723f2c32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:50.381698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f209fc3-1b8d-4434-9a54-0ffe723f2c32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:50.383274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f209fc3-1b8d-4434-9a54-0ffe723f2c32 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:50.392842 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1202/2380] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:04:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:04:52.547663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decef450-b218-487b-bd4a-1ab28a72a9d6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:52.548317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-decef450-b218-487b-bd4a-1ab28a72a9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:52.607024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:52.626480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decef450-b218-487b-bd4a-1ab28a72a9d6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:52.631077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:52.631714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:52.632010 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:52.632265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] PATH_INFO: `/v3/users/d0f11e586dc64a41a84248b4a4c894a9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:52.633411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f11e586dc64a41a84248b4a4c894a9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:52.635357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:52.708628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f11e586dc64a41a84248b4a4c894a9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:52.728557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f11e586dc64a41a84248b4a4c894a9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:52.729370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f11e586dc64a41a84248b4a4c894a9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:52.730052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f11e586dc64a41a84248b4a4c894a9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:52.738746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f11e586dc64a41a84248b4a4c894a9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:52.761979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1179/2381] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:04:52 2022] DELETE /identity/v3/users/d0f11e586dc64a41a84248b4a4c894a9 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:52.766833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1e508-c79d-4abf-b412-da8a9237b237 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:52.767376 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1e508-c79d-4abf-b412-da8a9237b237 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:52.767606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1e508-c79d-4abf-b412-da8a9237b237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:52.794982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-02e1e508-c79d-4abf-b412-da8a9237b237 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:52.796302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-02e1e508-c79d-4abf-b412-da8a9237b237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:52.847479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:52.871587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1203/2382] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:52.885887 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:52.887976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:52.952936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06380f13-780b-4eb8-ba3a-d1420cf63075 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:52.957214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16ada9-8f70-4dad-af9b-b22bb43cb8fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:52.958111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ada9-8f70-4dad-af9b-b22bb43cb8fb service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:52.958460 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ada9-8f70-4dad-af9b-b22bb43cb8fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:52.959028 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ada9-8f70-4dad-af9b-b22bb43cb8fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:52.970317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16ada9-8f70-4dad-af9b-b22bb43cb8fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:52.972014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16ada9-8f70-4dad-af9b-b22bb43cb8fb service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:52.983428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1180/2383] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:04:53.176657 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1e508-c79d-4abf-b412-da8a9237b237 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:53.178235 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-02e1e508-c79d-4abf-b412-da8a9237b237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:53.222980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1e508-c79d-4abf-b412-da8a9237b237 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:53.231616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:53.232452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:53.232741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:53.233034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] PATH_INFO: `/v3/projects/350221a9b0c14c70b92f48cae0d3a838` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:53.239360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] RBAC: Authorizing `identity:delete_project(project_id=350221a9b0c14c70b92f48cae0d3a838)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:53.241665 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:53.298037 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350221a9b0c14c70b92f48cae0d3a838', 'request_id': 'req-2e516f95-b875-40d0-850c-404a3b6f188d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:53.306530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350221a9b0c14c70b92f48cae0d3a838', 'request_id': 'req-2e516f95-b875-40d0-850c-404a3b6f188d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:53.313318 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350221a9b0c14c70b92f48cae0d3a838', 'request_id': 'req-2e516f95-b875-40d0-850c-404a3b6f188d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:53.326422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] The token cache is being invalidate because project 350221a9b0c14c70b92f48cae0d3a838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:04:53.326715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:53.328027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e516f95-b875-40d0-850c-404a3b6f188d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:04:53.329507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1204/2384] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:04:53 2022] DELETE /identity/v3/projects/350221a9b0c14c70b92f48cae0d3a838 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:53.334615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f26f1-4478-4ac9-9c0a-57e48658a897 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:53.335675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f26f1-4478-4ac9-9c0a-57e48658a897 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:53.336116 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f26f1-4478-4ac9-9c0a-57e48658a897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:53.357574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5e1f26f1-4478-4ac9-9c0a-57e48658a897 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:53.358784 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e1f26f1-4478-4ac9-9c0a-57e48658a897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:53.426892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1181/2385] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:53.434913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94abc0-c3a1-4d7c-80c1-83f46a7aab29 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:53.435180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94abc0-c3a1-4d7c-80c1-83f46a7aab29 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:53.435463 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94abc0-c3a1-4d7c-80c1-83f46a7aab29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:53.457009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9d94abc0-c3a1-4d7c-80c1-83f46a7aab29 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:53.458240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9d94abc0-c3a1-4d7c-80c1-83f46a7aab29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:53.467215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1205/2386] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:53.472586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f26f1-4478-4ac9-9c0a-57e48658a897 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:53.482243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f26f1-4478-4ac9-9c0a-57e48658a897 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:53.487195 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d09eae-012f-4b8c-9603-1b5d03f15489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:53.487811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d09eae-012f-4b8c-9603-1b5d03f15489 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:53.488094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d09eae-012f-4b8c-9603-1b5d03f15489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:53.488371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d09eae-012f-4b8c-9603-1b5d03f15489 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:53.489022 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d09eae-012f-4b8c-9603-1b5d03f15489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:04:53.489328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d09eae-012f-4b8c-9603-1b5d03f15489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:53.490918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d09eae-012f-4b8c-9603-1b5d03f15489 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:53.499904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:53.508550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94abc0-c3a1-4d7c-80c1-83f46a7aab29 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:53.510122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1182/2387] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:04:53.521428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94abc0-c3a1-4d7c-80c1-83f46a7aab29 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:53.526263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299aa80-d734-4e46-a01e-2ec6d2eabd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:53.526952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299aa80-d734-4e46-a01e-2ec6d2eabd34 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:53.527270 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299aa80-d734-4e46-a01e-2ec6d2eabd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:53.527543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299aa80-d734-4e46-a01e-2ec6d2eabd34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:53.528212 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299aa80-d734-4e46-a01e-2ec6d2eabd34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:53.531022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299aa80-d734-4e46-a01e-2ec6d2eabd34 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:53.567257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1206/2388] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:04:53 2022] POST /identity/v3/projects => generated 391 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:04:53.571843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d09eae-012f-4b8c-9603-1b5d03f15489 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:53.587197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d09eae-012f-4b8c-9603-1b5d03f15489 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:53.652615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a27e7-8bdd-484a-afd8-0f1edc887fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:53.653297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a27e7-8bdd-484a-afd8-0f1edc887fdb admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:53.653690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a27e7-8bdd-484a-afd8-0f1edc887fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:53.653818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a27e7-8bdd-484a-afd8-0f1edc887fdb admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:53.654456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a27e7-8bdd-484a-afd8-0f1edc887fdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:53.656571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a27e7-8bdd-484a-afd8-0f1edc887fdb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:53.683821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1183/2389] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:04:53 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:04:53.688767 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299aa80-d734-4e46-a01e-2ec6d2eabd34 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:53.702934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299aa80-d734-4e46-a01e-2ec6d2eabd34 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:53.708113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf7a96-e14f-4097-be71-a9e745e52d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:53.709113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf7a96-e14f-4097-be71-a9e745e52d33 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:53.709113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf7a96-e14f-4097-be71-a9e745e52d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:53.709407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf7a96-e14f-4097-be71-a9e745e52d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:53.711925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf7a96-e14f-4097-be71-a9e745e52d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:53.711925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf7a96-e14f-4097-be71-a9e745e52d33 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:53.725580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1207/2390] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:53.734141 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a27e7-8bdd-484a-afd8-0f1edc887fdb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:53.750678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a27e7-8bdd-484a-afd8-0f1edc887fdb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:53.762030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd18228-71f7-47ef-9cf5-b6dc60edb53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:53.762652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd18228-71f7-47ef-9cf5-b6dc60edb53c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:53.763742 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd18228-71f7-47ef-9cf5-b6dc60edb53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:53.764016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd18228-71f7-47ef-9cf5-b6dc60edb53c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:53.764662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd18228-71f7-47ef-9cf5-b6dc60edb53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:53.766394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd18228-71f7-47ef-9cf5-b6dc60edb53c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:53.781430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1184/2391] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:04:53.782744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf7a96-e14f-4097-be71-a9e745e52d33 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:53.793990 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf7a96-e14f-4097-be71-a9e745e52d33 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:53.799544 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85427121-210c-417f-a676-2bb391d3c65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:53.800204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85427121-210c-417f-a676-2bb391d3c65b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:53.800481 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85427121-210c-417f-a676-2bb391d3c65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:53.800897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85427121-210c-417f-a676-2bb391d3c65b admin admin] PATH_INFO: `/v3/projects/a917c701c9f24ce39b014b243eb5927c/users/b8b4550e454a40b591ea15564fc7815b/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:53.821697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85427121-210c-417f-a676-2bb391d3c65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a917c701c9f24ce39b014b243eb5927c, user_id=b8b4550e454a40b591ea15564fc7815b, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:53.824371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85427121-210c-417f-a676-2bb391d3c65b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:53.837975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1208/2392] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:04:53 2022] PUT /identity/v3/projects/a917c701c9f24ce39b014b243eb5927c/users/b8b4550e454a40b591ea15564fc7815b/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:04:53.854658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62afe1-d907-416f-a13a-3f91239edbfe None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:53.854945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62afe1-d907-416f-a13a-3f91239edbfe None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:53.855275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62afe1-d907-416f-a13a-3f91239edbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:53.884736 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6c62afe1-d907-416f-a13a-3f91239edbfe None None] MFA Rules not processed for user `b8b4550e454a40b591ea15564fc7815b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:04:53.886039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6c62afe1-d907-416f-a13a-3f91239edbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:53.956337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1185/2393] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:04:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:04:53.967336 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85427121-210c-417f-a676-2bb391d3c65b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:04:53.969064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-85427121-210c-417f-a676-2bb391d3c65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:04:54.039717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85427121-210c-417f-a676-2bb391d3c65b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:04:54.045874 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94435e63-321c-49bd-80db-542dc8b2ff47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:04:54.046497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94435e63-321c-49bd-80db-542dc8b2ff47 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:04:54.046778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94435e63-321c-49bd-80db-542dc8b2ff47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:04:54.047076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94435e63-321c-49bd-80db-542dc8b2ff47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:04:54.056824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94435e63-321c-49bd-80db-542dc8b2ff47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:04:54.064340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94435e63-321c-49bd-80db-542dc8b2ff47 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:04:54.072461 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1209/2394] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:04:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:05:03.953090 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:05:04.068029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:05:26.909444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62afe1-d907-416f-a13a-3f91239edbfe None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:05:26.972795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62afe1-d907-416f-a13a-3f91239edbfe None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:05:26.977248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:05:26.977876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:26.978166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:26.978444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] PATH_INFO: `/v3/users/b8b4550e454a40b591ea15564fc7815b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:26.979648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b4550e454a40b591ea15564fc7815b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:05:26.981565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:05:27.028354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b4550e454a40b591ea15564fc7815b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:05:27.040709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b4550e454a40b591ea15564fc7815b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:05:27.041469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b4550e454a40b591ea15564fc7815b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:05:27.042096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b4550e454a40b591ea15564fc7815b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:05:27.055391 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b4550e454a40b591ea15564fc7815b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:05:27.069243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1186/2395] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:05:26 2022] DELETE /identity/v3/users/b8b4550e454a40b591ea15564fc7815b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:05:27.073817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1484a0-35c2-4e7a-85f6-158d2e0ffb6e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:27.074333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1484a0-35c2-4e7a-85f6-158d2e0ffb6e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:27.074464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1484a0-35c2-4e7a-85f6-158d2e0ffb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:27.093635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ea1484a0-35c2-4e7a-85f6-158d2e0ffb6e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:05:27.094835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ea1484a0-35c2-4e7a-85f6-158d2e0ffb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:27.154282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1210/2396] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:05:27.163958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:05:27.165378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:27.223060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3c859-f03b-4e0a-868c-5f60cbdf55da admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:05:27.226799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade5e14-cc36-4664-a3cb-eab48ee20bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:05:27.227438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade5e14-cc36-4664-a3cb-eab48ee20bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:27.227745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade5e14-cc36-4664-a3cb-eab48ee20bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:27.228015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade5e14-cc36-4664-a3cb-eab48ee20bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:27.237923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade5e14-cc36-4664-a3cb-eab48ee20bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:05:27.239509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade5e14-cc36-4664-a3cb-eab48ee20bd8 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:05:27.253607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1187/2397] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:05:27.550942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1484a0-35c2-4e7a-85f6-158d2e0ffb6e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:05:27.552624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ea1484a0-35c2-4e7a-85f6-158d2e0ffb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:27.565616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1484a0-35c2-4e7a-85f6-158d2e0ffb6e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:05:27.571123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:05:27.571758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:27.572036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:27.572414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] PATH_INFO: `/v3/projects/a917c701c9f24ce39b014b243eb5927c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:27.573547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a917c701c9f24ce39b014b243eb5927c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:05:27.575532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:05:27.632311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a917c701c9f24ce39b014b243eb5927c', 'request_id': 'req-2c5f58ac-273e-4975-b682-6eea49ecde8c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:05:27.640518 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a917c701c9f24ce39b014b243eb5927c', 'request_id': 'req-2c5f58ac-273e-4975-b682-6eea49ecde8c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:05:27.647242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a917c701c9f24ce39b014b243eb5927c', 'request_id': 'req-2c5f58ac-273e-4975-b682-6eea49ecde8c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:05:27.660520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] The token cache is being invalidate because project a917c701c9f24ce39b014b243eb5927c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:05:27.660886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:05:27.661523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c5f58ac-273e-4975-b682-6eea49ecde8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:05:27.663161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1211/2398] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:05:27 2022] DELETE /identity/v3/projects/a917c701c9f24ce39b014b243eb5927c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:05:37.248742 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:05:37.657216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:06:04.640726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade5e14-cc36-4664-a3cb-eab48ee20bd8 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:04.642244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bade5e14-cc36-4664-a3cb-eab48ee20bd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:04.658929 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade5e14-cc36-4664-a3cb-eab48ee20bd8 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:04.716648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:04.717288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:04.717569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:04.717841 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] PATH_INFO: `/v3/users/2a7a359fc5974669a5dd8c59bd465b7e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:04.718953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a7a359fc5974669a5dd8c59bd465b7e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:04.721025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:04.766171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a7a359fc5974669a5dd8c59bd465b7e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:06:04.778550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a7a359fc5974669a5dd8c59bd465b7e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:06:04.779240 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a7a359fc5974669a5dd8c59bd465b7e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:06:04.779843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a7a359fc5974669a5dd8c59bd465b7e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:06:04.787263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a7a359fc5974669a5dd8c59bd465b7e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:06:04.799059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1188/2399] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:06:04 2022] DELETE /identity/v3/users/2a7a359fc5974669a5dd8c59bd465b7e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:06:04.803603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fae04-4727-4c48-a277-f2f69d0004bb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:04.804078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fae04-4727-4c48-a277-f2f69d0004bb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:04.804193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fae04-4727-4c48-a277-f2f69d0004bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:04.824126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-754fae04-4727-4c48-a277-f2f69d0004bb None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:06:04.825315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-754fae04-4727-4c48-a277-f2f69d0004bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:04.883260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1212/2400] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:06:04.892837 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:04.894922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:04.950489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3095f66-0e30-4565-aa34-0a73d5e84c78 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:04.954100 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a89984-3a6c-458d-94bf-defbc3dc39e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:04.954697 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a89984-3a6c-458d-94bf-defbc3dc39e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:04.954978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a89984-3a6c-458d-94bf-defbc3dc39e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:04.955352 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a89984-3a6c-458d-94bf-defbc3dc39e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:04.965138 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a89984-3a6c-458d-94bf-defbc3dc39e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:04.966614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a89984-3a6c-458d-94bf-defbc3dc39e1 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:04.975972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1189/2401] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:06:05.244114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fae04-4727-4c48-a277-f2f69d0004bb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:05.245576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-754fae04-4727-4c48-a277-f2f69d0004bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:05.259687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fae04-4727-4c48-a277-f2f69d0004bb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:05.264475 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:05.265324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:05.265628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:05.265900 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] PATH_INFO: `/v3/projects/1f7bebc3bd65445ea1f3c7a47e4dbaa2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:05.267060 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f7bebc3bd65445ea1f3c7a47e4dbaa2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:05.269016 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:05.324562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f7bebc3bd65445ea1f3c7a47e4dbaa2', 'request_id': 'req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:06:05.332852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f7bebc3bd65445ea1f3c7a47e4dbaa2', 'request_id': 'req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:06:05.339789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f7bebc3bd65445ea1f3c7a47e4dbaa2', 'request_id': 'req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:06:05.353497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] The token cache is being invalidate because project 1f7bebc3bd65445ea1f3c7a47e4dbaa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:06:05.353497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:06:05.353810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:06:05.355074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1213/2402] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:06:05 2022] DELETE /identity/v3/projects/1f7bebc3bd65445ea1f3c7a47e4dbaa2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:06:14.971880 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:06:15.349162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:06:37.125907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a89984-3a6c-458d-94bf-defbc3dc39e1 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:37.135162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a89984-3a6c-458d-94bf-defbc3dc39e1 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:37.184087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d523a-9c77-4c06-94df-41c81b3e1e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:37.184691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d523a-9c77-4c06-94df-41c81b3e1e93 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:37.184974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d523a-9c77-4c06-94df-41c81b3e1e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:37.185249 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d523a-9c77-4c06-94df-41c81b3e1e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:37.186966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1f0d523a-9c77-4c06-94df-41c81b3e1e93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:37.262986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d523a-9c77-4c06-94df-41c81b3e1e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:37.264515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d523a-9c77-4c06-94df-41c81b3e1e93 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:37.275476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1190/2403] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:06:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:06:37.386866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:37.388803 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:37.401747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8bcea-3021-4d2a-ad27-c15a7ade93d0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:37.405472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4afeb2-b626-4f69-8906-613bf6a133b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:37.406166 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4afeb2-b626-4f69-8906-613bf6a133b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:37.406511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4afeb2-b626-4f69-8906-613bf6a133b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:37.406794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4afeb2-b626-4f69-8906-613bf6a133b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:37.408537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-be4afeb2-b626-4f69-8906-613bf6a133b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:37.422463 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4afeb2-b626-4f69-8906-613bf6a133b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:37.423948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4afeb2-b626-4f69-8906-613bf6a133b8 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:37.436762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1214/2404] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:06:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:06:39.344687 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039c742-9a3a-479a-ab13-b2d46d3c65e7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:39.345167 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039c742-9a3a-479a-ab13-b2d46d3c65e7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:39.345261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039c742-9a3a-479a-ab13-b2d46d3c65e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:39.367403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6039c742-9a3a-479a-ab13-b2d46d3c65e7 None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:06:39.368557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6039c742-9a3a-479a-ab13-b2d46d3c65e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:39.423500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1191/2405] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:06:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:06:39.469931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4afeb2-b626-4f69-8906-613bf6a133b8 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:39.471369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-be4afeb2-b626-4f69-8906-613bf6a133b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:39.543147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4afeb2-b626-4f69-8906-613bf6a133b8 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:39.548168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54cb52-f78f-4592-beee-ca44c4326c18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:39.548773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54cb52-f78f-4592-beee-ca44c4326c18 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:39.549057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54cb52-f78f-4592-beee-ca44c4326c18 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:39.549330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54cb52-f78f-4592-beee-ca44c4326c18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:39.558512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54cb52-f78f-4592-beee-ca44c4326c18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:39.560130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54cb52-f78f-4592-beee-ca44c4326c18 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:39.568729 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1215/2406] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:06:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:06:48.068072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039c742-9a3a-479a-ab13-b2d46d3c65e7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:48.069630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6039c742-9a3a-479a-ab13-b2d46d3c65e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:48.087042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039c742-9a3a-479a-ab13-b2d46d3c65e7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:48.093127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328f582-9197-47bb-9383-dd4774d89208 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:48.093951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328f582-9197-47bb-9383-dd4774d89208 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:48.094235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328f582-9197-47bb-9383-dd4774d89208 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:48.094504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328f582-9197-47bb-9383-dd4774d89208 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:48.096319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1328f582-9197-47bb-9383-dd4774d89208 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:48.155224 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:06:48.171800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328f582-9197-47bb-9383-dd4774d89208 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:48.173554 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328f582-9197-47bb-9383-dd4774d89208 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:48.186958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1192/2407] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:06:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:06:49.564879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=38 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:06:54.002951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937b1c4-ed56-49d3-ba3b-7db6941ea595 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:54.003582 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937b1c4-ed56-49d3-ba3b-7db6941ea595 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:54.003582 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937b1c4-ed56-49d3-ba3b-7db6941ea595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:54.022811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5937b1c4-ed56-49d3-ba3b-7db6941ea595 None None] MFA Rules not processed for user `98e853439341466dbea3a535ca13d12b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:06:54.024029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5937b1c4-ed56-49d3-ba3b-7db6941ea595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:54.078620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1216/2408] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:06:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:06:54.700495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328f582-9197-47bb-9383-dd4774d89208 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:54.702099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1328f582-9197-47bb-9383-dd4774d89208 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:54.783679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328f582-9197-47bb-9383-dd4774d89208 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:54.789028 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c3903-ec83-41eb-84ae-6965a48f5ca0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:54.789640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c3903-ec83-41eb-84ae-6965a48f5ca0 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:54.789924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c3903-ec83-41eb-84ae-6965a48f5ca0 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:54.790200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c3903-ec83-41eb-84ae-6965a48f5ca0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:54.791935 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e75c3903-ec83-41eb-84ae-6965a48f5ca0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:54.804079 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c3903-ec83-41eb-84ae-6965a48f5ca0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:54.805744 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c3903-ec83-41eb-84ae-6965a48f5ca0 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:54.815767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1193/2409] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:06:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:06:58.522190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937b1c4-ed56-49d3-ba3b-7db6941ea595 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:58.532021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937b1c4-ed56-49d3-ba3b-7db6941ea595 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:58.536363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeab7be-24e1-480c-b81e-a8414b601b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:58.537081 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeab7be-24e1-480c-b81e-a8414b601b8f service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:58.537430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeab7be-24e1-480c-b81e-a8414b601b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:58.537774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeab7be-24e1-480c-b81e-a8414b601b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:58.539747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bfeab7be-24e1-480c-b81e-a8414b601b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:58.549720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeab7be-24e1-480c-b81e-a8414b601b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:58.551186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeab7be-24e1-480c-b81e-a8414b601b8f service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:58.559209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1217/2410] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:06:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:06:58.704570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbe3d2-db8a-431b-8333-a2173fb514bb None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:58.705520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbe3d2-db8a-431b-8333-a2173fb514bb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:58.705961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbe3d2-db8a-431b-8333-a2173fb514bb None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:58.708464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1194/2411] 10.222.0.86 () {54 vars in 921 bytes} [Mon Aug 29 19:06:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:06:58.715505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f7ed2-842b-46ca-9a99-b8673c1a0f39 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:58.715798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f7ed2-842b-46ca-9a99-b8673c1a0f39 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:58.716085 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f7ed2-842b-46ca-9a99-b8673c1a0f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:58.737620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-351f7ed2-842b-46ca-9a99-b8673c1a0f39 None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:06:58.738902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-351f7ed2-842b-46ca-9a99-b8673c1a0f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:58.755544 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1218/2412] 10.222.0.86 () {58 vars in 1018 bytes} [Mon Aug 29 19:06:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:06:58.764305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbe3d2-db8a-431b-8333-a2173fb514bb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:58.766832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c1fbe3d2-db8a-431b-8333-a2173fb514bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:58.839281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbe3d2-db8a-431b-8333-a2173fb514bb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:58.842509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c6e34-e6c1-42e9-917b-5da90138b45f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af69888b04c45b09e34d9ff7a0b564f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:58.843416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c6e34-e6c1-42e9-917b-5da90138b45f service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:58.843787 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c6e34-e6c1-42e9-917b-5da90138b45f service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:58.844152 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c6e34-e6c1-42e9-917b-5da90138b45f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:58.855468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c6e34-e6c1-42e9-917b-5da90138b45f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:58.857002 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c6e34-e6c1-42e9-917b-5da90138b45f service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:58.868077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1195/2413] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:06:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:06:58.943429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e668f-0890-48db-bc78-500caf796eee None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:58.943847 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e668f-0890-48db-bc78-500caf796eee None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:58.944190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e668f-0890-48db-bc78-500caf796eee None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:58.946047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1219/2414] 10.222.0.86 () {54 vars in 921 bytes} [Mon Aug 29 19:06:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:06:58.951081 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fe957-1405-4f56-bed0-e1ea42386ace None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:58.951440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fe957-1405-4f56-bed0-e1ea42386ace None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:58.951780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fe957-1405-4f56-bed0-e1ea42386ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:58.971938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-172fe957-1405-4f56-bed0-e1ea42386ace None None] MFA Rules not processed for user `5ec64944b34d4ef1bd666bb1b7e216da`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:06:58.973276 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-172fe957-1405-4f56-bed0-e1ea42386ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:58.980550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1196/2415] 10.222.0.86 () {58 vars in 1018 bytes} [Mon Aug 29 19:06:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:06:58.993131 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e668f-0890-48db-bc78-500caf796eee None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:59.005661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e668f-0890-48db-bc78-500caf796eee None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:59.009519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee3615-b3be-4888-8269-9d873ee3079e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:59.010375 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee3615-b3be-4888-8269-9d873ee3079e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:59.010712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee3615-b3be-4888-8269-9d873ee3079e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:59.011076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee3615-b3be-4888-8269-9d873ee3079e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:59.020236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee3615-b3be-4888-8269-9d873ee3079e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:59.021738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee3615-b3be-4888-8269-9d873ee3079e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:59.029770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1220/2416] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:06:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:06:59.118334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1684d6-e7f5-4859-a124-c41881d196da None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:59.118958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1684d6-e7f5-4859-a124-c41881d196da None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:59.118958 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1684d6-e7f5-4859-a124-c41881d196da None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:59.120948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1197/2417] 10.222.0.86 () {54 vars in 921 bytes} [Mon Aug 29 19:06:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:06:59.127407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8c6fd-be37-4ddf-9b41-052b8d21414b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:59.127720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8c6fd-be37-4ddf-9b41-052b8d21414b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:59.127998 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8c6fd-be37-4ddf-9b41-052b8d21414b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:59.152669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-00e8c6fd-be37-4ddf-9b41-052b8d21414b None None] MFA Rules not processed for user `045b22d046c549f2b53d2ed73b94f936`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:06:59.153897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-00e8c6fd-be37-4ddf-9b41-052b8d21414b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:59.164690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1221/2418] 10.222.0.86 () {58 vars in 1018 bytes} [Mon Aug 29 19:06:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:06:59.195802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1684d6-e7f5-4859-a124-c41881d196da None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:59.198055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fd1684d6-e7f5-4859-a124-c41881d196da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:59.214333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1684d6-e7f5-4859-a124-c41881d196da None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:59.219788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f492edd-1324-4e42-87fd-6ae31bb48760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:59.220944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f492edd-1324-4e42-87fd-6ae31bb48760 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:59.221297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f492edd-1324-4e42-87fd-6ae31bb48760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:59.221871 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f492edd-1324-4e42-87fd-6ae31bb48760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:59.240208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f492edd-1324-4e42-87fd-6ae31bb48760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:59.241853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f492edd-1324-4e42-87fd-6ae31bb48760 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:59.262342 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1198/2419] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:06:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:07:03.395985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8c6fd-be37-4ddf-9b41-052b8d21414b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:03.405701 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8c6fd-be37-4ddf-9b41-052b8d21414b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:03.409717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8709a2e-b471-4154-92ac-67b2527bf2ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:03.410324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8709a2e-b471-4154-92ac-67b2527bf2ff service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:03.410606 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8709a2e-b471-4154-92ac-67b2527bf2ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:03.410880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8709a2e-b471-4154-92ac-67b2527bf2ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:03.412740 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f8709a2e-b471-4154-92ac-67b2527bf2ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:03.423109 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8709a2e-b471-4154-92ac-67b2527bf2ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:03.425210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8709a2e-b471-4154-92ac-67b2527bf2ff service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:03.436969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1222/2420] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:05.842171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4b5fe-0bc5-4067-9222-56071cedc1ed None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:05.842606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4b5fe-0bc5-4067-9222-56071cedc1ed None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:05.842853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4b5fe-0bc5-4067-9222-56071cedc1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:05.866035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e6e4b5fe-0bc5-4067-9222-56071cedc1ed None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:05.867684 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e6e4b5fe-0bc5-4067-9222-56071cedc1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:05.886832 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1199/2421] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:07:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:05.904816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8709a2e-b471-4154-92ac-67b2527bf2ff service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:05.918666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8709a2e-b471-4154-92ac-67b2527bf2ff service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:05.927209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090fb0c-70a1-457f-8df0-e97411336b95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:05.927209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090fb0c-70a1-457f-8df0-e97411336b95 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:05.927209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090fb0c-70a1-457f-8df0-e97411336b95 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:05.927209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090fb0c-70a1-457f-8df0-e97411336b95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:05.938297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090fb0c-70a1-457f-8df0-e97411336b95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:05.940000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090fb0c-70a1-457f-8df0-e97411336b95 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:05.951101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1223/2422] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:07:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:07:15.860379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=77 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:15.945769 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=55 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:34.723978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4b5fe-0bc5-4067-9222-56071cedc1ed None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:34.725963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e6e4b5fe-0bc5-4067-9222-56071cedc1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:34.743310 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4b5fe-0bc5-4067-9222-56071cedc1ed None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:34.748759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396d5c1-9b7d-4fe8-8436-97f9e7595761 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:34.749364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396d5c1-9b7d-4fe8-8436-97f9e7595761 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:34.749640 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396d5c1-9b7d-4fe8-8436-97f9e7595761 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:34.749910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396d5c1-9b7d-4fe8-8436-97f9e7595761 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:34.751654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5396d5c1-9b7d-4fe8-8436-97f9e7595761 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:34.764030 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396d5c1-9b7d-4fe8-8436-97f9e7595761 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:34.765626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396d5c1-9b7d-4fe8-8436-97f9e7595761 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:34.776802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1200/2423] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:44.771622 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:49.324744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3149bfe-6289-4014-9e42-07df07ae237b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:49.325303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3149bfe-6289-4014-9e42-07df07ae237b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:49.325675 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3149bfe-6289-4014-9e42-07df07ae237b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:49.355042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b3149bfe-6289-4014-9e42-07df07ae237b None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:49.356545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b3149bfe-6289-4014-9e42-07df07ae237b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:49.364244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1224/2424] 10.222.0.86 () {58 vars in 967 bytes} [Mon Aug 29 19:07:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:49.388085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396d5c1-9b7d-4fe8-8436-97f9e7595761 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:49.400560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396d5c1-9b7d-4fe8-8436-97f9e7595761 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:49.405890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e55762-4979-495f-a23c-844736f0f35d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:49.406501 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e55762-4979-495f-a23c-844736f0f35d service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:49.406780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e55762-4979-495f-a23c-844736f0f35d service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:49.407094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e55762-4979-495f-a23c-844736f0f35d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:49.416369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e55762-4979-495f-a23c-844736f0f35d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:49.417944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e55762-4979-495f-a23c-844736f0f35d service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:49.427071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1201/2425] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:07:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:07:56.943113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3149bfe-6289-4014-9e42-07df07ae237b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:56.956855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3149bfe-6289-4014-9e42-07df07ae237b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:56.962744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209432d4-f4bc-456e-a439-99555e3f7330 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:56.963398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209432d4-f4bc-456e-a439-99555e3f7330 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:56.963691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209432d4-f4bc-456e-a439-99555e3f7330 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:56.963964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209432d4-f4bc-456e-a439-99555e3f7330 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:56.965681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-209432d4-f4bc-456e-a439-99555e3f7330 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:56.979092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209432d4-f4bc-456e-a439-99555e3f7330 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:56.980703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209432d4-f4bc-456e-a439-99555e3f7330 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:56.991378 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1225/2426] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:59.424423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:06.986815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=9 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:17.162834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e55762-4979-495f-a23c-844736f0f35d service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:17.175670 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e55762-4979-495f-a23c-844736f0f35d service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:17.181164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2648a4a-407f-4548-95c9-ba6a12efa987 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:17.181757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2648a4a-407f-4548-95c9-ba6a12efa987 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:17.182039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2648a4a-407f-4548-95c9-ba6a12efa987 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:17.182315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2648a4a-407f-4548-95c9-ba6a12efa987 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:17.184028 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f2648a4a-407f-4548-95c9-ba6a12efa987 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:17.241791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2648a4a-407f-4548-95c9-ba6a12efa987 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:17.243693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2648a4a-407f-4548-95c9-ba6a12efa987 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:17.252678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1202/2427] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:27.248499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=28 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:27.589243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209432d4-f4bc-456e-a439-99555e3f7330 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:27.600364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209432d4-f4bc-456e-a439-99555e3f7330 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:27.606089 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfc42e-3b70-41d7-a439-17462138d328 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:27.606691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfc42e-3b70-41d7-a439-17462138d328 service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:27.606969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfc42e-3b70-41d7-a439-17462138d328 service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:27.607315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfc42e-3b70-41d7-a439-17462138d328 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:27.609054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e5dfc42e-3b70-41d7-a439-17462138d328 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:27.620843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfc42e-3b70-41d7-a439-17462138d328 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:27.622334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfc42e-3b70-41d7-a439-17462138d328 service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:27.632335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1226/2428] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:27.653622 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2648a4a-407f-4548-95c9-ba6a12efa987 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:27.655074 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f2648a4a-407f-4548-95c9-ba6a12efa987 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:27.724238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2648a4a-407f-4548-95c9-ba6a12efa987 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:27.728765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbc355-c29b-47b3-8869-00a20e0029f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:27.729396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbc355-c29b-47b3-8869-00a20e0029f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:27.729678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbc355-c29b-47b3-8869-00a20e0029f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:27.729952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbc355-c29b-47b3-8869-00a20e0029f4 admin admin] PATH_INFO: `/v3/users/98e853439341466dbea3a535ca13d12b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:27.731249 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbc355-c29b-47b3-8869-00a20e0029f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e853439341466dbea3a535ca13d12b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:27.733236 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbc355-c29b-47b3-8869-00a20e0029f4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:27.782187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7cdbc355-c29b-47b3-8869-00a20e0029f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e853439341466dbea3a535ca13d12b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:27.794296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7cdbc355-c29b-47b3-8869-00a20e0029f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e853439341466dbea3a535ca13d12b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:27.794979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7cdbc355-c29b-47b3-8869-00a20e0029f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e853439341466dbea3a535ca13d12b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:27.795637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7cdbc355-c29b-47b3-8869-00a20e0029f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e853439341466dbea3a535ca13d12b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:27.805075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7cdbc355-c29b-47b3-8869-00a20e0029f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e853439341466dbea3a535ca13d12b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:27.818284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1203/2429] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:27 2022] DELETE /identity/v3/users/98e853439341466dbea3a535ca13d12b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:27.990744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfc42e-3b70-41d7-a439-17462138d328 service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.001558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfc42e-3b70-41d7-a439-17462138d328 service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.059627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.060253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.060528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.060796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] PATH_INFO: `/v3/projects/7918fb7f8aea4f58857307485bc09ec6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.062323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7918fb7f8aea4f58857307485bc09ec6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.064254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.110906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7918fb7f8aea4f58857307485bc09ec6', 'request_id': 'req-30b13b37-4f9b-4035-9738-30ddf0d31180'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:28.118033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7918fb7f8aea4f58857307485bc09ec6', 'request_id': 'req-30b13b37-4f9b-4035-9738-30ddf0d31180'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:28.124765 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7918fb7f8aea4f58857307485bc09ec6', 'request_id': 'req-30b13b37-4f9b-4035-9738-30ddf0d31180'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:28.135322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] The token cache is being invalidate because project 7918fb7f8aea4f58857307485bc09ec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:28.135607 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:28.136231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-30b13b37-4f9b-4035-9738-30ddf0d31180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:28.137687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1227/2430] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:27 2022] DELETE /identity/v3/projects/7918fb7f8aea4f58857307485bc09ec6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:28.142769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b673ef-b9e7-43f7-b116-53ef8963a21e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.143097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b673ef-b9e7-43f7-b116-53ef8963a21e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.143380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b673ef-b9e7-43f7-b116-53ef8963a21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.163648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a6b673ef-b9e7-43f7-b116-53ef8963a21e None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:28.164862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a6b673ef-b9e7-43f7-b116-53ef8963a21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:28.223131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1204/2431] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:28.231453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e952d-ea4d-402c-a050-45842407b7fa None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.231739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e952d-ea4d-402c-a050-45842407b7fa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.232036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e952d-ea4d-402c-a050-45842407b7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.251822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ed1e952d-ea4d-402c-a050-45842407b7fa None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:28.253088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ed1e952d-ea4d-402c-a050-45842407b7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:28.261195 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1228/2432] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:28.265329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b673ef-b9e7-43f7-b116-53ef8963a21e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.276058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b673ef-b9e7-43f7-b116-53ef8963a21e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.280805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e47de-c171-40ff-8835-ca0f1ffd6708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.281406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e47de-c171-40ff-8835-ca0f1ffd6708 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.281683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e47de-c171-40ff-8835-ca0f1ffd6708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.281956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e47de-c171-40ff-8835-ca0f1ffd6708 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.282601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e47de-c171-40ff-8835-ca0f1ffd6708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:28.282908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e47de-c171-40ff-8835-ca0f1ffd6708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.284531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e47de-c171-40ff-8835-ca0f1ffd6708 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.296694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1205/2433] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:28.300947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e952d-ea4d-402c-a050-45842407b7fa None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.310150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e952d-ea4d-402c-a050-45842407b7fa None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.314744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fe737-f247-4cc5-9b85-693a1e6f5e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.315346 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe737-f247-4cc5-9b85-693a1e6f5e87 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.315619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe737-f247-4cc5-9b85-693a1e6f5e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.315886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe737-f247-4cc5-9b85-693a1e6f5e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.316502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fe737-f247-4cc5-9b85-693a1e6f5e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.318377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fe737-f247-4cc5-9b85-693a1e6f5e87 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.345237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1229/2434] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:28 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:28.349281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e47de-c171-40ff-8835-ca0f1ffd6708 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.360381 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e47de-c171-40ff-8835-ca0f1ffd6708 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.364724 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:28.420773 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caba6a4-eb2e-4846-a3bd-a609ca9d1919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.421370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caba6a4-eb2e-4846-a3bd-a609ca9d1919 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.421645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caba6a4-eb2e-4846-a3bd-a609ca9d1919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.421915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caba6a4-eb2e-4846-a3bd-a609ca9d1919 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.422541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caba6a4-eb2e-4846-a3bd-a609ca9d1919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.424471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caba6a4-eb2e-4846-a3bd-a609ca9d1919 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.444629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1206/2435] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:28 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:28.449507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fe737-f247-4cc5-9b85-693a1e6f5e87 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.459539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fe737-f247-4cc5-9b85-693a1e6f5e87 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.464663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5fcbe-e9cf-4795-b462-0fa1d565097f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.465257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5fcbe-e9cf-4795-b462-0fa1d565097f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.465532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5fcbe-e9cf-4795-b462-0fa1d565097f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.465801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5fcbe-e9cf-4795-b462-0fa1d565097f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.466433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5fcbe-e9cf-4795-b462-0fa1d565097f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.468492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5fcbe-e9cf-4795-b462-0fa1d565097f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.481981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1230/2436] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:28.486418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caba6a4-eb2e-4846-a3bd-a609ca9d1919 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.496737 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caba6a4-eb2e-4846-a3bd-a609ca9d1919 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.501469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bba65-4682-44b7-ad60-f3e7c1bcf854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.502063 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bba65-4682-44b7-ad60-f3e7c1bcf854 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.502341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bba65-4682-44b7-ad60-f3e7c1bcf854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.502616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bba65-4682-44b7-ad60-f3e7c1bcf854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.503321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bba65-4682-44b7-ad60-f3e7c1bcf854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.504905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bba65-4682-44b7-ad60-f3e7c1bcf854 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.515930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1207/2437] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:28.519946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5fcbe-e9cf-4795-b462-0fa1d565097f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.530913 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5fcbe-e9cf-4795-b462-0fa1d565097f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.535550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e83cf-bba8-41bb-9c09-d9f976920f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.536199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e83cf-bba8-41bb-9c09-d9f976920f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.536478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e83cf-bba8-41bb-9c09-d9f976920f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.536747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e83cf-bba8-41bb-9c09-d9f976920f8f admin admin] PATH_INFO: `/v3/projects/5d70fdff5c5a443b9f0ae1a24dfffcd3/users/0547f15d00f043b99de10f3db7348c62/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.551631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e83cf-bba8-41bb-9c09-d9f976920f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d70fdff5c5a443b9f0ae1a24dfffcd3, user_id=0547f15d00f043b99de10f3db7348c62, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.554260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e83cf-bba8-41bb-9c09-d9f976920f8f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.566811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1231/2438] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:28 2022] PUT /identity/v3/projects/5d70fdff5c5a443b9f0ae1a24dfffcd3/users/0547f15d00f043b99de10f3db7348c62/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:28.572696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27895004-e1ad-447e-8e22-a1f63aa6718f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.572986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27895004-e1ad-447e-8e22-a1f63aa6718f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.573264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27895004-e1ad-447e-8e22-a1f63aa6718f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.592844 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-27895004-e1ad-447e-8e22-a1f63aa6718f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:28.594049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-27895004-e1ad-447e-8e22-a1f63aa6718f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:28.651168 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1208/2439] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:28.660855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e83cf-bba8-41bb-9c09-d9f976920f8f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.662642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-785e83cf-bba8-41bb-9c09-d9f976920f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:28.676609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:28.722393 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e83cf-bba8-41bb-9c09-d9f976920f8f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.726039 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c575-1d5b-4062-a20a-aebff51bad4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.726640 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c575-1d5b-4062-a20a-aebff51bad4b service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.726917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c575-1d5b-4062-a20a-aebff51bad4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.727330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c575-1d5b-4062-a20a-aebff51bad4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.736050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01c575-1d5b-4062-a20a-aebff51bad4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.737403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01c575-1d5b-4062-a20a-aebff51bad4b service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.746942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1232/2440] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:32.981106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8e2a-cac6-4285-9aaa-0a83669ae316 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:32.981650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8e2a-cac6-4285-9aaa-0a83669ae316 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:32.981650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8e2a-cac6-4285-9aaa-0a83669ae316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:33.013060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2fee8e2a-cac6-4285-9aaa-0a83669ae316 None None] MFA Rules not processed for user `0547f15d00f043b99de10f3db7348c62`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:33.014294 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2fee8e2a-cac6-4285-9aaa-0a83669ae316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:33.090746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1209/2441] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:08:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:33.095288 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c575-1d5b-4062-a20a-aebff51bad4b service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:33.111453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c575-1d5b-4062-a20a-aebff51bad4b service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:33.116983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34998e-c993-469a-807f-b37bc207c816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:33.117605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34998e-c993-469a-807f-b37bc207c816 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:33.117882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34998e-c993-469a-807f-b37bc207c816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:33.118193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34998e-c993-469a-807f-b37bc207c816 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:33.118840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34998e-c993-469a-807f-b37bc207c816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:33.120805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34998e-c993-469a-807f-b37bc207c816 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:33.152619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1233/2442] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:33 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:33.156955 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee8e2a-cac6-4285-9aaa-0a83669ae316 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:33.169061 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee8e2a-cac6-4285-9aaa-0a83669ae316 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:33.181859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:33.230456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0385a4-749b-4e98-a991-7f2b1c6a20ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:33.231107 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0385a4-749b-4e98-a991-7f2b1c6a20ed admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:33.231386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0385a4-749b-4e98-a991-7f2b1c6a20ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:33.231801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0385a4-749b-4e98-a991-7f2b1c6a20ed admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:33.232437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0385a4-749b-4e98-a991-7f2b1c6a20ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:33.234414 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0385a4-749b-4e98-a991-7f2b1c6a20ed admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:33.257412 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1210/2443] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:33 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:33.262110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34998e-c993-469a-807f-b37bc207c816 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:33.273036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34998e-c993-469a-807f-b37bc207c816 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:33.278491 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab8086-1940-40d9-956a-00364eeeb162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:33.279342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab8086-1940-40d9-956a-00364eeeb162 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:33.279799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab8086-1940-40d9-956a-00364eeeb162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:33.280249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab8086-1940-40d9-956a-00364eeeb162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:33.281059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab8086-1940-40d9-956a-00364eeeb162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:33.282837 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab8086-1940-40d9-956a-00364eeeb162 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:33.294435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1234/2444] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:33.298892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0385a4-749b-4e98-a991-7f2b1c6a20ed admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:33.309980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0385a4-749b-4e98-a991-7f2b1c6a20ed admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:33.314647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5891257-8ea2-452a-998a-f2d58fbdbb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:33.315350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5891257-8ea2-452a-998a-f2d58fbdbb3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:33.315628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5891257-8ea2-452a-998a-f2d58fbdbb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:33.315928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5891257-8ea2-452a-998a-f2d58fbdbb3c admin admin] PATH_INFO: `/v3/projects/91b7f611c9ad43d4ac9f13506c654f76/users/81d3c2ac625a4cffbdde0da4be64a65e/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:33.333938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5891257-8ea2-452a-998a-f2d58fbdbb3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b7f611c9ad43d4ac9f13506c654f76, user_id=81d3c2ac625a4cffbdde0da4be64a65e, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:33.337397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5891257-8ea2-452a-998a-f2d58fbdbb3c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:33.355164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1211/2445] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:33 2022] PUT /identity/v3/projects/91b7f611c9ad43d4ac9f13506c654f76/users/81d3c2ac625a4cffbdde0da4be64a65e/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:33.359790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab8086-1940-40d9-956a-00364eeeb162 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:33.371248 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab8086-1940-40d9-956a-00364eeeb162 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:33.431710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f271772-b555-4312-8811-c868cb86cffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:33.432311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f271772-b555-4312-8811-c868cb86cffc admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:33.432587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f271772-b555-4312-8811-c868cb86cffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:33.432856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f271772-b555-4312-8811-c868cb86cffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:33.433478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f271772-b555-4312-8811-c868cb86cffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:33.435086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f271772-b555-4312-8811-c868cb86cffc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:33.444937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1235/2446] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:33.449937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5891257-8ea2-452a-998a-f2d58fbdbb3c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:33.460372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5891257-8ea2-452a-998a-f2d58fbdbb3c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:33.465379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291ea99-73f8-4de0-a629-dd6b7e4a30d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:33.466315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291ea99-73f8-4de0-a629-dd6b7e4a30d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:33.466776 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291ea99-73f8-4de0-a629-dd6b7e4a30d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:33.467327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291ea99-73f8-4de0-a629-dd6b7e4a30d3 admin admin] PATH_INFO: `/v3/projects/91b7f611c9ad43d4ac9f13506c654f76/users/81d3c2ac625a4cffbdde0da4be64a65e/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:33.470655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291ea99-73f8-4de0-a629-dd6b7e4a30d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b7f611c9ad43d4ac9f13506c654f76, user_id=81d3c2ac625a4cffbdde0da4be64a65e, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:33.475196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291ea99-73f8-4de0-a629-dd6b7e4a30d3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:33.486916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1212/2447] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:33 2022] PUT /identity/v3/projects/91b7f611c9ad43d4ac9f13506c654f76/users/81d3c2ac625a4cffbdde0da4be64a65e/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:34.058625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f271772-b555-4312-8811-c868cb86cffc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:34.060786 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1f271772-b555-4312-8811-c868cb86cffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:34.170914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f271772-b555-4312-8811-c868cb86cffc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:34.193412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:34.271870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad068fb-83d4-460a-936d-872f9228a8b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:34.272753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad068fb-83d4-460a-936d-872f9228a8b6 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:34.273183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad068fb-83d4-460a-936d-872f9228a8b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:34.273599 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad068fb-83d4-460a-936d-872f9228a8b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:34.286425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dad068fb-83d4-460a-936d-872f9228a8b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:34.368625 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad068fb-83d4-460a-936d-872f9228a8b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:34.370394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad068fb-83d4-460a-936d-872f9228a8b6 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:34.384683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1236/2448] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:38.258338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa0c4-7842-4d98-a58e-bae003231864 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.258893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa0c4-7842-4d98-a58e-bae003231864 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.258988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa0c4-7842-4d98-a58e-bae003231864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.294154 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-52dfa0c4-7842-4d98-a58e-bae003231864 None None] MFA Rules not processed for user `81d3c2ac625a4cffbdde0da4be64a65e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:38.295647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-52dfa0c4-7842-4d98-a58e-bae003231864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.420799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1213/2449] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:08:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:38.445115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad068fb-83d4-460a-936d-872f9228a8b6 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.446887 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dad068fb-83d4-460a-936d-872f9228a8b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.468895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad068fb-83d4-460a-936d-872f9228a8b6 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.486502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622cd8e-ff4e-4af8-9e6f-83df8bf9b016 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.498706 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622cd8e-ff4e-4af8-9e6f-83df8bf9b016 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.499657 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622cd8e-ff4e-4af8-9e6f-83df8bf9b016 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.500095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622cd8e-ff4e-4af8-9e6f-83df8bf9b016 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.602088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622cd8e-ff4e-4af8-9e6f-83df8bf9b016 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.605351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622cd8e-ff4e-4af8-9e6f-83df8bf9b016 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.616167 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1237/2450] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:41.526953 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb5e8e-95fe-4410-a6e8-78e93abae552 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:41.527580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb5e8e-95fe-4410-a6e8-78e93abae552 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:41.527580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb5e8e-95fe-4410-a6e8-78e93abae552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:41.560102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6dbb5e8e-95fe-4410-a6e8-78e93abae552 None None] MFA Rules not processed for user `962284e9a353430881bc119e5f80a890`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:41.561401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6dbb5e8e-95fe-4410-a6e8-78e93abae552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:41.621198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:41.635170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1214/2451] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:08:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:48.610863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=76 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:51.629302 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:04.810915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce6f09-04ac-4015-ab17-f9ed523b2aed None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:04.811455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce6f09-04ac-4015-ab17-f9ed523b2aed None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:04.811578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce6f09-04ac-4015-ab17-f9ed523b2aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:04.831564 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-51ce6f09-04ac-4015-ab17-f9ed523b2aed None None] MFA Rules not processed for user `0547f15d00f043b99de10f3db7348c62`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:04.832838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-51ce6f09-04ac-4015-ab17-f9ed523b2aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:04.840018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1238/2452] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:09:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:14.825899 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:23.395891 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb5e8e-95fe-4410-a6e8-78e93abae552 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:23.454228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb5e8e-95fe-4410-a6e8-78e93abae552 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:23.458205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a064c2-1d53-4883-8356-c3dced7eb0ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:23.458814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a064c2-1d53-4883-8356-c3dced7eb0ea service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:23.459125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a064c2-1d53-4883-8356-c3dced7eb0ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:23.459400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a064c2-1d53-4883-8356-c3dced7eb0ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:23.468611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a064c2-1d53-4883-8356-c3dced7eb0ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:23.470217 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a064c2-1d53-4883-8356-c3dced7eb0ea service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:23.480157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1215/2453] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:09:26.846447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce6f09-04ac-4015-ab17-f9ed523b2aed None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:26.923880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce6f09-04ac-4015-ab17-f9ed523b2aed None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:26.929676 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4299d9-f187-4e81-8bdf-8207f1b35c7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:26.930295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4299d9-f187-4e81-8bdf-8207f1b35c7b service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:26.930624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4299d9-f187-4e81-8bdf-8207f1b35c7b service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:26.930902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4299d9-f187-4e81-8bdf-8207f1b35c7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:26.932678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3a4299d9-f187-4e81-8bdf-8207f1b35c7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:26.970967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4299d9-f187-4e81-8bdf-8207f1b35c7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:26.972666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4299d9-f187-4e81-8bdf-8207f1b35c7b service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:26.985008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1239/2454] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:30.471235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc601dfd-daba-422c-8b3c-21fd933022a5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:30.471716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc601dfd-daba-422c-8b3c-21fd933022a5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:30.471810 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc601dfd-daba-422c-8b3c-21fd933022a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:30.492528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fc601dfd-daba-422c-8b3c-21fd933022a5 None None] MFA Rules not processed for user `dd013ba83e944ecfa1a2d0a8a0d55101`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:30.493769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fc601dfd-daba-422c-8b3c-21fd933022a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:30.546499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1216/2455] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:09:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:36.979857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=39 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:40.540054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=51 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:11.786806 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c21c-56b9-4d29-a544-e166558ea1a1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:11.787400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c21c-56b9-4d29-a544-e166558ea1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:11.788264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c21c-56b9-4d29-a544-e166558ea1a1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:11.789794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1240/2456] 10.222.0.86 () {54 vars in 961 bytes} [Mon Aug 29 19:10:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:10:11.797780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec5367-743d-4f54-94e5-8de73faf7adc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:11.798338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec5367-743d-4f54-94e5-8de73faf7adc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:11.799433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec5367-743d-4f54-94e5-8de73faf7adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:11.901284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c4ec5367-743d-4f54-94e5-8de73faf7adc None None] MFA Rules not processed for user `27f771f20b734b679d5553d29a8a3884`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:11.903277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c4ec5367-743d-4f54-94e5-8de73faf7adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:11.993653 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1217/2457] 10.222.0.86 () {58 vars in 1058 bytes} [Mon Aug 29 19:10:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:12.002747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005c21c-56b9-4d29-a544-e166558ea1a1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:12.030179 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005c21c-56b9-4d29-a544-e166558ea1a1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:12.045280 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd38d1a-ff84-4a70-9ded-6db286990c5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:12.046275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd38d1a-ff84-4a70-9ded-6db286990c5f service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:12.047194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd38d1a-ff84-4a70-9ded-6db286990c5f service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:12.047746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd38d1a-ff84-4a70-9ded-6db286990c5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:12.050035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7bd38d1a-ff84-4a70-9ded-6db286990c5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:12.063827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd38d1a-ff84-4a70-9ded-6db286990c5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:12.065584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd38d1a-ff84-4a70-9ded-6db286990c5f service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:12.076716 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1241/2458] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:16.407258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb491b33-f7d8-4bf0-94ce-13ab56b9cbc2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.407805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb491b33-f7d8-4bf0-94ce-13ab56b9cbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.407805 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb491b33-f7d8-4bf0-94ce-13ab56b9cbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.427306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fb491b33-f7d8-4bf0-94ce-13ab56b9cbc2 None None] MFA Rules not processed for user `b114570010d94eea955b7b3f1fa97c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:16.429112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb491b33-f7d8-4bf0-94ce-13ab56b9cbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:16.435963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1218/2459] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:10:16.742843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd38d1a-ff84-4a70-9ded-6db286990c5f service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:16.744276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7bd38d1a-ff84-4a70-9ded-6db286990c5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:16.779446 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd38d1a-ff84-4a70-9ded-6db286990c5f service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:16.783078 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d3673-aa17-49e4-bfe8-4c6bfd4c1f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:16.783833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d3673-aa17-49e4-bfe8-4c6bfd4c1f8b service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.784121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d3673-aa17-49e4-bfe8-4c6bfd4c1f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.784391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d3673-aa17-49e4-bfe8-4c6bfd4c1f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.786113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a82d3673-aa17-49e4-bfe8-4c6bfd4c1f8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:16.797045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d3673-aa17-49e4-bfe8-4c6bfd4c1f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:16.798456 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d3673-aa17-49e4-bfe8-4c6bfd4c1f8b service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:16.807187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1242/2460] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:25.980851 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb491b33-f7d8-4bf0-94ce-13ab56b9cbc2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:25.991648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb491b33-f7d8-4bf0-94ce-13ab56b9cbc2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:25.995246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852d8fc-f419-49e0-9fd1-4a06381d7148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:25.995882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852d8fc-f419-49e0-9fd1-4a06381d7148 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:25.996190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852d8fc-f419-49e0-9fd1-4a06381d7148 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:25.996459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852d8fc-f419-49e0-9fd1-4a06381d7148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:25.998208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2852d8fc-f419-49e0-9fd1-4a06381d7148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:26.090522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852d8fc-f419-49e0-9fd1-4a06381d7148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:26.092008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852d8fc-f419-49e0-9fd1-4a06381d7148 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:26.112696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1219/2461] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:26.802954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=21 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:29.135337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d3673-aa17-49e4-bfe8-4c6bfd4c1f8b service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:29.148855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d3673-aa17-49e4-bfe8-4c6bfd4c1f8b service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:29.153997 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94715069-2ecc-4b91-95a7-0a0f759a8e0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:29.154690 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94715069-2ecc-4b91-95a7-0a0f759a8e0c service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:29.155085 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94715069-2ecc-4b91-95a7-0a0f759a8e0c service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:29.155430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94715069-2ecc-4b91-95a7-0a0f759a8e0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:29.157250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-94715069-2ecc-4b91-95a7-0a0f759a8e0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:29.170813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94715069-2ecc-4b91-95a7-0a0f759a8e0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:29.173264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94715069-2ecc-4b91-95a7-0a0f759a8e0c service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:29.188147 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1243/2462] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:29.219353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852d8fc-f419-49e0-9fd1-4a06381d7148 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:29.221072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2852d8fc-f419-49e0-9fd1-4a06381d7148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:29.234502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:29.241417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852d8fc-f419-49e0-9fd1-4a06381d7148 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:29.246587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8878e-c96b-4655-8d30-9d05b143e900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:29.247358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8878e-c96b-4655-8d30-9d05b143e900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:29.247705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8878e-c96b-4655-8d30-9d05b143e900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:29.248050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8878e-c96b-4655-8d30-9d05b143e900 admin admin] PATH_INFO: `/v3/users/962284e9a353430881bc119e5f80a890` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:29.249308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8878e-c96b-4655-8d30-9d05b143e900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=962284e9a353430881bc119e5f80a890)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:29.251367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8878e-c96b-4655-8d30-9d05b143e900 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:29.319714 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ec8878e-c96b-4655-8d30-9d05b143e900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '962284e9a353430881bc119e5f80a890'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:29.343694 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ec8878e-c96b-4655-8d30-9d05b143e900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '962284e9a353430881bc119e5f80a890'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:29.344514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ec8878e-c96b-4655-8d30-9d05b143e900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '962284e9a353430881bc119e5f80a890'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:29.345119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ec8878e-c96b-4655-8d30-9d05b143e900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '962284e9a353430881bc119e5f80a890'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:29.355794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ec8878e-c96b-4655-8d30-9d05b143e900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '962284e9a353430881bc119e5f80a890'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:29.370072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1220/2463] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:10:29 2022] DELETE /identity/v3/users/962284e9a353430881bc119e5f80a890 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:29.548882 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94715069-2ecc-4b91-95a7-0a0f759a8e0c service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:29.562661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94715069-2ecc-4b91-95a7-0a0f759a8e0c service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:29.667699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:29.668358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:29.668644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:29.668974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] PATH_INFO: `/v3/projects/4f32b60f5fa8427386bafa1a91f4e608` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:29.670140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f32b60f5fa8427386bafa1a91f4e608)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:29.672236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:29.743585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f32b60f5fa8427386bafa1a91f4e608', 'request_id': 'req-90d946ad-b496-4505-99ac-83567a9632bb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:29.753403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f32b60f5fa8427386bafa1a91f4e608', 'request_id': 'req-90d946ad-b496-4505-99ac-83567a9632bb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:29.761163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f32b60f5fa8427386bafa1a91f4e608', 'request_id': 'req-90d946ad-b496-4505-99ac-83567a9632bb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:29.773949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] The token cache is being invalidate because project 4f32b60f5fa8427386bafa1a91f4e608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:10:29.774242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:29.774966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-90d946ad-b496-4505-99ac-83567a9632bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:29.776535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1244/2464] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:10:29 2022] DELETE /identity/v3/projects/4f32b60f5fa8427386bafa1a91f4e608 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:29.781538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b67928-f943-4a91-950b-904637294a87 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:29.781831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b67928-f943-4a91-950b-904637294a87 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:29.782167 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b67928-f943-4a91-950b-904637294a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:29.842658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b6b67928-f943-4a91-950b-904637294a87 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:29.843873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b6b67928-f943-4a91-950b-904637294a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:29.914054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1221/2465] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:29.922403 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93bdbc-6783-4a89-906e-9bdb728efdb7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:29.922948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93bdbc-6783-4a89-906e-9bdb728efdb7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:29.923336 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93bdbc-6783-4a89-906e-9bdb728efdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:29.948153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bc93bdbc-6783-4a89-906e-9bdb728efdb7 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:29.949570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bc93bdbc-6783-4a89-906e-9bdb728efdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:29.959094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1245/2466] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:29.963584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b67928-f943-4a91-950b-904637294a87 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:29.974087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b67928-f943-4a91-950b-904637294a87 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:29.982223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40818df7-efa7-40d9-958c-e3c96b142aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:29.983021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40818df7-efa7-40d9-958c-e3c96b142aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:29.983403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40818df7-efa7-40d9-958c-e3c96b142aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:29.984263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40818df7-efa7-40d9-958c-e3c96b142aa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:29.984606 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40818df7-efa7-40d9-958c-e3c96b142aa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:10:29.985007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40818df7-efa7-40d9-958c-e3c96b142aa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:29.986830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40818df7-efa7-40d9-958c-e3c96b142aa2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:30.003191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1222/2467] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:10:30.005134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93bdbc-6783-4a89-906e-9bdb728efdb7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:30.015284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93bdbc-6783-4a89-906e-9bdb728efdb7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:30.020551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d88d7e-8eb4-4c5a-8533-582d684b32c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:30.021219 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d88d7e-8eb4-4c5a-8533-582d684b32c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:30.021602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d88d7e-8eb4-4c5a-8533-582d684b32c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:30.021938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d88d7e-8eb4-4c5a-8533-582d684b32c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:30.022724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d88d7e-8eb4-4c5a-8533-582d684b32c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:30.025412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d88d7e-8eb4-4c5a-8533-582d684b32c2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:30.054631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1246/2468] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:10:30 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:10:30.058500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40818df7-efa7-40d9-958c-e3c96b142aa2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:30.070422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40818df7-efa7-40d9-958c-e3c96b142aa2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:30.119537 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:30.140656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f14ac-f7dd-4566-86a2-5280410d536b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:30.141331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f14ac-f7dd-4566-86a2-5280410d536b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:30.141622 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f14ac-f7dd-4566-86a2-5280410d536b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:30.141934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f14ac-f7dd-4566-86a2-5280410d536b admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:30.142648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f14ac-f7dd-4566-86a2-5280410d536b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:30.144782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f14ac-f7dd-4566-86a2-5280410d536b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:30.172530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1223/2469] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:10:30 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:10:30.177150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d88d7e-8eb4-4c5a-8533-582d684b32c2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:30.192584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d88d7e-8eb4-4c5a-8533-582d684b32c2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:30.200196 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ce19c-0394-4995-9482-0c72c4056094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:30.200525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ce19c-0394-4995-9482-0c72c4056094 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:30.200825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ce19c-0394-4995-9482-0c72c4056094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:30.201102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ce19c-0394-4995-9482-0c72c4056094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:30.201761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ce19c-0394-4995-9482-0c72c4056094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:30.203411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ce19c-0394-4995-9482-0c72c4056094 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:30.219205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1247/2470] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:10:30.220580 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f14ac-f7dd-4566-86a2-5280410d536b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:30.233970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f14ac-f7dd-4566-86a2-5280410d536b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:30.238798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c344391-1583-425d-a0a3-6c6fdb5935b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:30.239454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c344391-1583-425d-a0a3-6c6fdb5935b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:30.239735 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c344391-1583-425d-a0a3-6c6fdb5935b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:30.240008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c344391-1583-425d-a0a3-6c6fdb5935b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:30.241191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c344391-1583-425d-a0a3-6c6fdb5935b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:30.242839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c344391-1583-425d-a0a3-6c6fdb5935b3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:30.256587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1224/2471] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:10:30.258043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ce19c-0394-4995-9482-0c72c4056094 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:30.269049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ce19c-0394-4995-9482-0c72c4056094 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:30.274681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa036e-5430-41de-9e22-2852eff770e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:30.275475 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa036e-5430-41de-9e22-2852eff770e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:30.275835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa036e-5430-41de-9e22-2852eff770e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:30.276129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa036e-5430-41de-9e22-2852eff770e8 admin admin] PATH_INFO: `/v3/projects/8e483c67faaf4b7e952919ce5174da35/users/f61d3852c17d468dbbd897af74e7dd8f/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:30.292578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa036e-5430-41de-9e22-2852eff770e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e483c67faaf4b7e952919ce5174da35, user_id=f61d3852c17d468dbbd897af74e7dd8f, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:30.295412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa036e-5430-41de-9e22-2852eff770e8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:30.311863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1248/2472] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:10:30 2022] PUT /identity/v3/projects/8e483c67faaf4b7e952919ce5174da35/users/f61d3852c17d468dbbd897af74e7dd8f/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:30.325129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0636c5-4889-4fa9-bee8-788b46e85efe None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:30.325426 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0636c5-4889-4fa9-bee8-788b46e85efe None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:30.325706 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0636c5-4889-4fa9-bee8-788b46e85efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:30.362895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7d0636c5-4889-4fa9-bee8-788b46e85efe None None] MFA Rules not processed for user `f61d3852c17d468dbbd897af74e7dd8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:30.364248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7d0636c5-4889-4fa9-bee8-788b46e85efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:30.434307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1225/2473] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:30.446208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa036e-5430-41de-9e22-2852eff770e8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:30.448512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d9aa036e-5430-41de-9e22-2852eff770e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:30.464836 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:30.551295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa036e-5430-41de-9e22-2852eff770e8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:30.556825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83a600-f031-4a2e-9f33-9624b4bd7b76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:30.557447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83a600-f031-4a2e-9f33-9624b4bd7b76 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:30.557719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83a600-f031-4a2e-9f33-9624b4bd7b76 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:30.557987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83a600-f031-4a2e-9f33-9624b4bd7b76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:30.566441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83a600-f031-4a2e-9f33-9624b4bd7b76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:30.568040 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83a600-f031-4a2e-9f33-9624b4bd7b76 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:30.580087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1249/2474] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:30.817157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0636c5-4889-4fa9-bee8-788b46e85efe None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:30.882400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0636c5-4889-4fa9-bee8-788b46e85efe None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:30.887613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:30.888178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:30.888453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:30.888720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] PATH_INFO: `/v3/users/f61d3852c17d468dbbd897af74e7dd8f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:30.889855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f61d3852c17d468dbbd897af74e7dd8f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:30.891836 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:30.940793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f61d3852c17d468dbbd897af74e7dd8f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:30.946258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:30.959504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f61d3852c17d468dbbd897af74e7dd8f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:30.959692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f61d3852c17d468dbbd897af74e7dd8f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:30.960289 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f61d3852c17d468dbbd897af74e7dd8f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:30.969454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f61d3852c17d468dbbd897af74e7dd8f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:30.982027 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1226/2475] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:10:30 2022] DELETE /identity/v3/users/f61d3852c17d468dbbd897af74e7dd8f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:30.986775 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d3a35-ab7e-4349-8313-a231f59bb093 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:30.987122 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d3a35-ab7e-4349-8313-a231f59bb093 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:30.987553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d3a35-ab7e-4349-8313-a231f59bb093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:31.007914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-db1d3a35-ab7e-4349-8313-a231f59bb093 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:31.009131 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db1d3a35-ab7e-4349-8313-a231f59bb093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:31.082492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1250/2476] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:31.092427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:31.093935 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:31.174777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf0147-ea56-429c-aca3-34993a1cf633 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:31.179251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1a9c1-238f-4634-81dc-dc6e305728c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:31.179853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1a9c1-238f-4634-81dc-dc6e305728c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:31.180130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1a9c1-238f-4634-81dc-dc6e305728c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:31.180563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1a9c1-238f-4634-81dc-dc6e305728c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:31.189411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1a9c1-238f-4634-81dc-dc6e305728c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:31.190815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1a9c1-238f-4634-81dc-dc6e305728c8 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:31.200420 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1227/2477] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:31.475537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d3a35-ab7e-4349-8313-a231f59bb093 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:31.477038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db1d3a35-ab7e-4349-8313-a231f59bb093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:31.490778 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d3a35-ab7e-4349-8313-a231f59bb093 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:31.495370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:31.496000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:31.496273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:31.496543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] PATH_INFO: `/v3/projects/8e483c67faaf4b7e952919ce5174da35` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:31.498136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e483c67faaf4b7e952919ce5174da35)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:31.500312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:31.548739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e483c67faaf4b7e952919ce5174da35', 'request_id': 'req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:31.555906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e483c67faaf4b7e952919ce5174da35', 'request_id': 'req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:31.562211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e483c67faaf4b7e952919ce5174da35', 'request_id': 'req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:31.572364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] The token cache is being invalidate because project 8e483c67faaf4b7e952919ce5174da35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:10:31.572906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:31.573528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e63aa93e-c3b1-4f6e-8fcd-92a94c4d0d61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:31.575260 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1251/2478] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:10:31 2022] DELETE /identity/v3/projects/8e483c67faaf4b7e952919ce5174da35 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:31.580121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d524a80-01f1-48c0-bc8a-ae23b7b33574 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:31.580626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d524a80-01f1-48c0-bc8a-ae23b7b33574 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:31.580752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d524a80-01f1-48c0-bc8a-ae23b7b33574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:31.600915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3d524a80-01f1-48c0-bc8a-ae23b7b33574 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:31.603090 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3d524a80-01f1-48c0-bc8a-ae23b7b33574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:31.663788 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1228/2479] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:31.672072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b391baf-ec05-4760-81d6-f28e19759a34 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:31.672362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b391baf-ec05-4760-81d6-f28e19759a34 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:31.672657 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b391baf-ec05-4760-81d6-f28e19759a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:31.694739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4b391baf-ec05-4760-81d6-f28e19759a34 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:31.695996 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4b391baf-ec05-4760-81d6-f28e19759a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:31.704520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1252/2480] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:10:31.708783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d524a80-01f1-48c0-bc8a-ae23b7b33574 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:31.718706 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d524a80-01f1-48c0-bc8a-ae23b7b33574 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:31.723494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d80fab-0850-4e28-bb9c-87e5d0c7dab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:31.724107 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d80fab-0850-4e28-bb9c-87e5d0c7dab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:31.724378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d80fab-0850-4e28-bb9c-87e5d0c7dab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:31.724666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d80fab-0850-4e28-bb9c-87e5d0c7dab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:31.725338 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d80fab-0850-4e28-bb9c-87e5d0c7dab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:10:31.725663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d80fab-0850-4e28-bb9c-87e5d0c7dab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:31.727859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d80fab-0850-4e28-bb9c-87e5d0c7dab8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:31.742676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1229/2481] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:10:31.747103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b391baf-ec05-4760-81d6-f28e19759a34 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:31.756484 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b391baf-ec05-4760-81d6-f28e19759a34 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:31.761151 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f705a7-2ea6-4bee-8f9c-0b8ab20f6761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:31.761749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f705a7-2ea6-4bee-8f9c-0b8ab20f6761 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:31.762026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f705a7-2ea6-4bee-8f9c-0b8ab20f6761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:31.762295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f705a7-2ea6-4bee-8f9c-0b8ab20f6761 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:31.762930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f705a7-2ea6-4bee-8f9c-0b8ab20f6761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:31.764856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f705a7-2ea6-4bee-8f9c-0b8ab20f6761 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:31.779989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:31.794047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1253/2482] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:10:31 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:10:31.798417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d80fab-0850-4e28-bb9c-87e5d0c7dab8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:31.809008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d80fab-0850-4e28-bb9c-87e5d0c7dab8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:31.865241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa4e0b-c300-42fa-8549-22ce973b5741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:31.867074 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa4e0b-c300-42fa-8549-22ce973b5741 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:31.867357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa4e0b-c300-42fa-8549-22ce973b5741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:31.867631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa4e0b-c300-42fa-8549-22ce973b5741 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:31.868247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa4e0b-c300-42fa-8549-22ce973b5741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:31.870155 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa4e0b-c300-42fa-8549-22ce973b5741 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:31.878007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:31.892204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1230/2483] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:10:31 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:10:31.896596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f705a7-2ea6-4bee-8f9c-0b8ab20f6761 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:31.907397 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f705a7-2ea6-4bee-8f9c-0b8ab20f6761 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:31.912246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e50e5-e973-4cc6-be20-93ebb25dc6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:31.912880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e50e5-e973-4cc6-be20-93ebb25dc6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:31.913160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e50e5-e973-4cc6-be20-93ebb25dc6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:31.913433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e50e5-e973-4cc6-be20-93ebb25dc6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:31.914150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e50e5-e973-4cc6-be20-93ebb25dc6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:31.915749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e50e5-e973-4cc6-be20-93ebb25dc6f8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:31.925824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1254/2484] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:10:31.929660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa4e0b-c300-42fa-8549-22ce973b5741 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:31.939715 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa4e0b-c300-42fa-8549-22ce973b5741 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:31.944896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72971d-40ff-4c39-bda1-d1739c26450b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:31.945486 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72971d-40ff-4c39-bda1-d1739c26450b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:31.945766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72971d-40ff-4c39-bda1-d1739c26450b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:31.946037 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72971d-40ff-4c39-bda1-d1739c26450b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:31.946668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72971d-40ff-4c39-bda1-d1739c26450b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:31.948561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72971d-40ff-4c39-bda1-d1739c26450b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:31.958830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1231/2485] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:10:31.963653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e50e5-e973-4cc6-be20-93ebb25dc6f8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:31.973702 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e50e5-e973-4cc6-be20-93ebb25dc6f8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:31.978958 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dd717-dc2f-4b1e-8baa-626abaedc3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:31.979654 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dd717-dc2f-4b1e-8baa-626abaedc3a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:31.979937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dd717-dc2f-4b1e-8baa-626abaedc3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:31.980211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dd717-dc2f-4b1e-8baa-626abaedc3a6 admin admin] PATH_INFO: `/v3/projects/0d7655eb5d0b4f7fbd14d3f4aa9816f6/users/93b78488a06842bdb20139f3a20f6a59/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:31.996458 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342dd717-dc2f-4b1e-8baa-626abaedc3a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d7655eb5d0b4f7fbd14d3f4aa9816f6, user_id=93b78488a06842bdb20139f3a20f6a59, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:31.999133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342dd717-dc2f-4b1e-8baa-626abaedc3a6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:32.013265 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1255/2486] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:10:31 2022] PUT /identity/v3/projects/0d7655eb5d0b4f7fbd14d3f4aa9816f6/users/93b78488a06842bdb20139f3a20f6a59/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:32.025479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5542edd-0453-4dd1-a1f1-220e289e2e6c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:32.025775 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5542edd-0453-4dd1-a1f1-220e289e2e6c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:32.026196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5542edd-0453-4dd1-a1f1-220e289e2e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:32.053793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f5542edd-0453-4dd1-a1f1-220e289e2e6c None None] MFA Rules not processed for user `93b78488a06842bdb20139f3a20f6a59`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:32.055018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f5542edd-0453-4dd1-a1f1-220e289e2e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:32.118025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1232/2487] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:32.129077 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dd717-dc2f-4b1e-8baa-626abaedc3a6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:32.130790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-342dd717-dc2f-4b1e-8baa-626abaedc3a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:32.192886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dd717-dc2f-4b1e-8baa-626abaedc3a6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:32.196957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11398938-8f2b-4a5d-a0dc-abae2d786ea7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af69888b04c45b09e34d9ff7a0b564f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:32.197808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11398938-8f2b-4a5d-a0dc-abae2d786ea7 service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:32.198097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11398938-8f2b-4a5d-a0dc-abae2d786ea7 service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:32.198370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11398938-8f2b-4a5d-a0dc-abae2d786ea7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:32.206652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11398938-8f2b-4a5d-a0dc-abae2d786ea7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:32.208142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11398938-8f2b-4a5d-a0dc-abae2d786ea7 service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:32.218127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1256/2488] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:42.115110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:42.213679 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:58.256018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5542edd-0453-4dd1-a1f1-220e289e2e6c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:58.318090 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5542edd-0453-4dd1-a1f1-220e289e2e6c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:58.322542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:58.323467 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:58.323748 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:58.324021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] PATH_INFO: `/v3/users/93b78488a06842bdb20139f3a20f6a59` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:58.325117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b78488a06842bdb20139f3a20f6a59)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:58.327061 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:58.376421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b78488a06842bdb20139f3a20f6a59'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:58.388529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b78488a06842bdb20139f3a20f6a59'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:58.389148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b78488a06842bdb20139f3a20f6a59'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:58.389757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b78488a06842bdb20139f3a20f6a59'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:58.396795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b78488a06842bdb20139f3a20f6a59'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:58.409025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1233/2489] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:10:58 2022] DELETE /identity/v3/users/93b78488a06842bdb20139f3a20f6a59 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:58.413997 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd41d-f8db-4616-80f9-bf3ed063899d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:58.414488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd41d-f8db-4616-80f9-bf3ed063899d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:58.414603 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd41d-f8db-4616-80f9-bf3ed063899d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:58.433499 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d3bbd41d-f8db-4616-80f9-bf3ed063899d None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:58.434705 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3bbd41d-f8db-4616-80f9-bf3ed063899d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:58.492316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1257/2490] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:58.501954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:58.503376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:58.559654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb732d9-20b0-4e59-83ed-856b6e084634 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:58.563350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22927ce-1707-4a82-b04c-8c8008a12bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:58.564034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22927ce-1707-4a82-b04c-8c8008a12bdd service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:58.564380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22927ce-1707-4a82-b04c-8c8008a12bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:58.564657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22927ce-1707-4a82-b04c-8c8008a12bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:58.574688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22927ce-1707-4a82-b04c-8c8008a12bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:58.576153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22927ce-1707-4a82-b04c-8c8008a12bdd service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:58.585426 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1234/2491] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:58.853564 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbd41d-f8db-4616-80f9-bf3ed063899d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:58.855206 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3bbd41d-f8db-4616-80f9-bf3ed063899d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:58.869875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbd41d-f8db-4616-80f9-bf3ed063899d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:58.874456 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:58.875776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:58.876245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:58.876529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] PATH_INFO: `/v3/projects/0d7655eb5d0b4f7fbd14d3f4aa9816f6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:58.877670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d7655eb5d0b4f7fbd14d3f4aa9816f6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:58.879718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:58.926489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d7655eb5d0b4f7fbd14d3f4aa9816f6', 'request_id': 'req-e4bdb735-ac73-41b2-ae4b-7e85e008b635'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:58.933132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d7655eb5d0b4f7fbd14d3f4aa9816f6', 'request_id': 'req-e4bdb735-ac73-41b2-ae4b-7e85e008b635'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:58.939505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d7655eb5d0b4f7fbd14d3f4aa9816f6', 'request_id': 'req-e4bdb735-ac73-41b2-ae4b-7e85e008b635'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:58.950513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] The token cache is being invalidate because project 0d7655eb5d0b4f7fbd14d3f4aa9816f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:10:58.950799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:58.951550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4bdb735-ac73-41b2-ae4b-7e85e008b635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:58.953182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1258/2492] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:10:58 2022] DELETE /identity/v3/projects/0d7655eb5d0b4f7fbd14d3f4aa9816f6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:58.957988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f52ce-37a2-4370-bca7-e5a20a740d1c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:58.958278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f52ce-37a2-4370-bca7-e5a20a740d1c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:58.959153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f52ce-37a2-4370-bca7-e5a20a740d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:58.978309 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-215f52ce-37a2-4370-bca7-e5a20a740d1c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:58.979537 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-215f52ce-37a2-4370-bca7-e5a20a740d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:58.996131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:59.040797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1235/2493] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:59.048720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f82df-7cac-4f4a-b4df-5c86c9c64f12 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.049009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f82df-7cac-4f4a-b4df-5c86c9c64f12 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.049291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f82df-7cac-4f4a-b4df-5c86c9c64f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.069624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0e0f82df-7cac-4f4a-b4df-5c86c9c64f12 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:59.070834 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0e0f82df-7cac-4f4a-b4df-5c86c9c64f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:59.079164 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1259/2494] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:59.084040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f52ce-37a2-4370-bca7-e5a20a740d1c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:59.093670 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f52ce-37a2-4370-bca7-e5a20a740d1c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:59.098216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f5477-c70a-4448-9768-8298902466a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:59.098811 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f5477-c70a-4448-9768-8298902466a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.099156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f5477-c70a-4448-9768-8298902466a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.099433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f5477-c70a-4448-9768-8298902466a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.100075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f5477-c70a-4448-9768-8298902466a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:10:59.100392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f5477-c70a-4448-9768-8298902466a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:59.101974 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f5477-c70a-4448-9768-8298902466a0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:59.113696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1236/2495] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:10:59.118028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f82df-7cac-4f4a-b4df-5c86c9c64f12 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:59.127326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f82df-7cac-4f4a-b4df-5c86c9c64f12 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:59.132519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ce4c6-3cc9-48d4-96a2-6bd7c72de5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:59.133112 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ce4c6-3cc9-48d4-96a2-6bd7c72de5ab admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.133392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ce4c6-3cc9-48d4-96a2-6bd7c72de5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.133662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ce4c6-3cc9-48d4-96a2-6bd7c72de5ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.134303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ce4c6-3cc9-48d4-96a2-6bd7c72de5ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:59.136233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ce4c6-3cc9-48d4-96a2-6bd7c72de5ab admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:59.164444 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1260/2496] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:10:59 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:10:59.169413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f5477-c70a-4448-9768-8298902466a0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:59.179319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f5477-c70a-4448-9768-8298902466a0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:59.238639 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521abe3-40af-486d-b30c-58819cb8cf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:59.239185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521abe3-40af-486d-b30c-58819cb8cf20 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.239462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521abe3-40af-486d-b30c-58819cb8cf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.240250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521abe3-40af-486d-b30c-58819cb8cf20 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.240872 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521abe3-40af-486d-b30c-58819cb8cf20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:59.242765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521abe3-40af-486d-b30c-58819cb8cf20 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:59.265210 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1237/2497] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:10:59 2022] POST /identity/v3/users => generated 279 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:10:59.269620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ce4c6-3cc9-48d4-96a2-6bd7c72de5ab admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:59.280074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ce4c6-3cc9-48d4-96a2-6bd7c72de5ab admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:59.284945 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a42c7-06c0-4c36-94e4-3f13d95081aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:59.285540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a42c7-06c0-4c36-94e4-3f13d95081aa admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.285822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a42c7-06c0-4c36-94e4-3f13d95081aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.286096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a42c7-06c0-4c36-94e4-3f13d95081aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.286730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a42c7-06c0-4c36-94e4-3f13d95081aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:59.288354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a42c7-06c0-4c36-94e4-3f13d95081aa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:59.298181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1261/2498] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:10:59.302618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521abe3-40af-486d-b30c-58819cb8cf20 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:59.312397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521abe3-40af-486d-b30c-58819cb8cf20 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:59.316942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4badb9-84c2-48f7-ace6-43b08cc7b4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:59.317522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4badb9-84c2-48f7-ace6-43b08cc7b4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.318373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4badb9-84c2-48f7-ace6-43b08cc7b4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.318659 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4badb9-84c2-48f7-ace6-43b08cc7b4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.319323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4badb9-84c2-48f7-ace6-43b08cc7b4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:59.320889 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4badb9-84c2-48f7-ace6-43b08cc7b4f9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:59.331385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1238/2499] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:10:59.335345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a42c7-06c0-4c36-94e4-3f13d95081aa admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:59.345355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a42c7-06c0-4c36-94e4-3f13d95081aa admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:59.349933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a0718-45ef-4ab1-bf1d-b3c6e8c0633b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:59.350578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a0718-45ef-4ab1-bf1d-b3c6e8c0633b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.350895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a0718-45ef-4ab1-bf1d-b3c6e8c0633b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.351210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a0718-45ef-4ab1-bf1d-b3c6e8c0633b admin admin] PATH_INFO: `/v3/projects/ee7a4db5e3c64d0da502d94fc485267d/users/a794a684ab104ab4aed122e555a101e4/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.366101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a0718-45ef-4ab1-bf1d-b3c6e8c0633b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7a4db5e3c64d0da502d94fc485267d, user_id=a794a684ab104ab4aed122e555a101e4, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:59.368929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a0718-45ef-4ab1-bf1d-b3c6e8c0633b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:59.381649 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1262/2500] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:10:59 2022] PUT /identity/v3/projects/ee7a4db5e3c64d0da502d94fc485267d/users/a794a684ab104ab4aed122e555a101e4/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:59.392052 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce93bcd-86cb-4b0b-814a-019e84851410 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.392334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce93bcd-86cb-4b0b-814a-019e84851410 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.392607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce93bcd-86cb-4b0b-814a-019e84851410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.420572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fce93bcd-86cb-4b0b-814a-019e84851410 None None] MFA Rules not processed for user `a794a684ab104ab4aed122e555a101e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:59.421778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fce93bcd-86cb-4b0b-814a-019e84851410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:59.486793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1239/2501] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:59.520031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a0718-45ef-4ab1-bf1d-b3c6e8c0633b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:59.522361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5d0a0718-45ef-4ab1-bf1d-b3c6e8c0633b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:59.546322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:59.605343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a0718-45ef-4ab1-bf1d-b3c6e8c0633b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:59.608978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d4b57-45bb-4340-8eda-835acf50af18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:59.609577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4b57-45bb-4340-8eda-835acf50af18 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.609855 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4b57-45bb-4340-8eda-835acf50af18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.610128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4b57-45bb-4340-8eda-835acf50af18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.618373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4b57-45bb-4340-8eda-835acf50af18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:59.619823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4b57-45bb-4340-8eda-835acf50af18 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:59.628662 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1263/2502] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:59.644499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce93bcd-86cb-4b0b-814a-019e84851410 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:59.664534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:59.713774 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce93bcd-86cb-4b0b-814a-019e84851410 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:59.718421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:59.719080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.719364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.719664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] PATH_INFO: `/v3/users/a794a684ab104ab4aed122e555a101e4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.720797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a794a684ab104ab4aed122e555a101e4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:59.722814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:59.770204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a794a684ab104ab4aed122e555a101e4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:59.782357 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a794a684ab104ab4aed122e555a101e4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:59.783127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a794a684ab104ab4aed122e555a101e4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:59.783725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a794a684ab104ab4aed122e555a101e4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:59.790827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a794a684ab104ab4aed122e555a101e4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:59.803368 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1240/2503] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:10:59 2022] DELETE /identity/v3/users/a794a684ab104ab4aed122e555a101e4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:59.807894 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2cea-985b-45d7-ab60-f33f8a6ad79c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.808186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2cea-985b-45d7-ab60-f33f8a6ad79c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.808465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2cea-985b-45d7-ab60-f33f8a6ad79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.828159 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d8ad2cea-985b-45d7-ab60-f33f8a6ad79c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:59.829357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d8ad2cea-985b-45d7-ab60-f33f8a6ad79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:59.888241 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1264/2504] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:59.898064 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:59.899522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:59.956234 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9c5a8-6e85-4e6c-9e12-1f25120e9aaa admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:59.960358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9bcac-1852-4479-a6b6-8fd14ab41903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:59.960955 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9bcac-1852-4479-a6b6-8fd14ab41903 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:59.961238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9bcac-1852-4479-a6b6-8fd14ab41903 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:59.961506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9bcac-1852-4479-a6b6-8fd14ab41903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:59.970166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9bcac-1852-4479-a6b6-8fd14ab41903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:59.971619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9bcac-1852-4479-a6b6-8fd14ab41903 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:59.981355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1241/2505] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:00.259383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad2cea-985b-45d7-ab60-f33f8a6ad79c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:00.261256 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d8ad2cea-985b-45d7-ab60-f33f8a6ad79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:00.277127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad2cea-985b-45d7-ab60-f33f8a6ad79c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:00.281924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:00.282559 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:00.282896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:00.283237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] PATH_INFO: `/v3/projects/ee7a4db5e3c64d0da502d94fc485267d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:00.284523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7a4db5e3c64d0da502d94fc485267d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:00.286434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:00.334738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7a4db5e3c64d0da502d94fc485267d', 'request_id': 'req-5c64edbd-748a-4350-8a94-4345de6d5599'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:00.342342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7a4db5e3c64d0da502d94fc485267d', 'request_id': 'req-5c64edbd-748a-4350-8a94-4345de6d5599'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:00.348697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7a4db5e3c64d0da502d94fc485267d', 'request_id': 'req-5c64edbd-748a-4350-8a94-4345de6d5599'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:00.359594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] The token cache is being invalidate because project ee7a4db5e3c64d0da502d94fc485267d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:11:00.359884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:00.360515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c64edbd-748a-4350-8a94-4345de6d5599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:00.361994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1265/2506] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:11:00 2022] DELETE /identity/v3/projects/ee7a4db5e3c64d0da502d94fc485267d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:00.367609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d349dff-32b9-42d5-8ea5-20ee6a52a64c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:00.368078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d349dff-32b9-42d5-8ea5-20ee6a52a64c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:00.368180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d349dff-32b9-42d5-8ea5-20ee6a52a64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:00.388595 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7d349dff-32b9-42d5-8ea5-20ee6a52a64c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:00.389835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7d349dff-32b9-42d5-8ea5-20ee6a52a64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:00.422139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:00.449997 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1242/2507] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:00.459944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336c04a-acfa-4403-a79b-1b946442b2ff None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:00.460233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336c04a-acfa-4403-a79b-1b946442b2ff None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:00.460508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336c04a-acfa-4403-a79b-1b946442b2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:00.480420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7336c04a-acfa-4403-a79b-1b946442b2ff None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:00.481626 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7336c04a-acfa-4403-a79b-1b946442b2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:00.489916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1266/2508] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:00.494268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d349dff-32b9-42d5-8ea5-20ee6a52a64c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:00.504154 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d349dff-32b9-42d5-8ea5-20ee6a52a64c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:00.509405 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c1c15-91da-4973-899b-88e6db990d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:00.510017 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c1c15-91da-4973-899b-88e6db990d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:00.510489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c1c15-91da-4973-899b-88e6db990d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:00.510942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c1c15-91da-4973-899b-88e6db990d60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:00.512059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c1c15-91da-4973-899b-88e6db990d60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:11:00.512543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c1c15-91da-4973-899b-88e6db990d60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:00.515157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c1c15-91da-4973-899b-88e6db990d60 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:00.529068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1243/2509] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:11:00.533396 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336c04a-acfa-4403-a79b-1b946442b2ff None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:00.543135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336c04a-acfa-4403-a79b-1b946442b2ff None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:00.547839 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d67bd-fb60-4cae-844a-2339e213bf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:00.548624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d67bd-fb60-4cae-844a-2339e213bf19 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:00.548925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d67bd-fb60-4cae-844a-2339e213bf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:00.549225 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d67bd-fb60-4cae-844a-2339e213bf19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:00.549852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d67bd-fb60-4cae-844a-2339e213bf19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:00.551867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d67bd-fb60-4cae-844a-2339e213bf19 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:00.572801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:00.586719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1267/2510] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:11:00 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:11:00.590930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c1c15-91da-4973-899b-88e6db990d60 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:00.602948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c1c15-91da-4973-899b-88e6db990d60 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:00.661171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdb0d9-4ba9-4fff-b08b-b355283c84f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:00.661851 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdb0d9-4ba9-4fff-b08b-b355283c84f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:00.662131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdb0d9-4ba9-4fff-b08b-b355283c84f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:00.662406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdb0d9-4ba9-4fff-b08b-b355283c84f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:00.663080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdb0d9-4ba9-4fff-b08b-b355283c84f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:00.664989 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdb0d9-4ba9-4fff-b08b-b355283c84f2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:00.686816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1244/2511] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:11:00 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:11:00.691093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d67bd-fb60-4cae-844a-2339e213bf19 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:00.702817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d67bd-fb60-4cae-844a-2339e213bf19 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:00.708209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cec18-067a-4179-816e-43b92f334af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:00.709471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cec18-067a-4179-816e-43b92f334af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:00.709750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cec18-067a-4179-816e-43b92f334af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:00.710022 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cec18-067a-4179-816e-43b92f334af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:00.710653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cec18-067a-4179-816e-43b92f334af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:00.712285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cec18-067a-4179-816e-43b92f334af5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:00.722242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1268/2512] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:11:00.726234 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdb0d9-4ba9-4fff-b08b-b355283c84f2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:00.736384 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdb0d9-4ba9-4fff-b08b-b355283c84f2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:00.741500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baac3bd-b2ff-47e5-8a67-0abf71e697a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:00.742089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baac3bd-b2ff-47e5-8a67-0abf71e697a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:00.742371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baac3bd-b2ff-47e5-8a67-0abf71e697a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:00.742644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baac3bd-b2ff-47e5-8a67-0abf71e697a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:00.743323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baac3bd-b2ff-47e5-8a67-0abf71e697a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:00.744903 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baac3bd-b2ff-47e5-8a67-0abf71e697a7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:00.756176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1245/2513] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:11:00.760293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cec18-067a-4179-816e-43b92f334af5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:00.770414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cec18-067a-4179-816e-43b92f334af5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:00.775041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99692b-3be7-4cc3-9200-ba840aa92f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:00.775711 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99692b-3be7-4cc3-9200-ba840aa92f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:00.775990 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99692b-3be7-4cc3-9200-ba840aa92f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:00.776264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99692b-3be7-4cc3-9200-ba840aa92f63 admin admin] PATH_INFO: `/v3/projects/7fd9313005a1492dacc7b73f7b09c179/users/d52349a5b2d84daeacad5003726019b5/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:00.790452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99692b-3be7-4cc3-9200-ba840aa92f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd9313005a1492dacc7b73f7b09c179, user_id=d52349a5b2d84daeacad5003726019b5, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:00.793191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99692b-3be7-4cc3-9200-ba840aa92f63 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:00.806257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1269/2514] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:11:00 2022] PUT /identity/v3/projects/7fd9313005a1492dacc7b73f7b09c179/users/d52349a5b2d84daeacad5003726019b5/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:00.816474 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3445d1-4215-4c35-ac37-fd0386f557e9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:00.816762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3445d1-4215-4c35-ac37-fd0386f557e9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:00.817085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3445d1-4215-4c35-ac37-fd0386f557e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:00.847229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ea3445d1-4215-4c35-ac37-fd0386f557e9 None None] MFA Rules not processed for user `d52349a5b2d84daeacad5003726019b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:00.848407 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ea3445d1-4215-4c35-ac37-fd0386f557e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:00.915012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1246/2515] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:00.919066 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99692b-3be7-4cc3-9200-ba840aa92f63 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:00.993228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99692b-3be7-4cc3-9200-ba840aa92f63 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:00.997929 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fd6e7-168f-48d9-88a6-9988fa568415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:00.998531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fd6e7-168f-48d9-88a6-9988fa568415 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:00.998812 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fd6e7-168f-48d9-88a6-9988fa568415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:00.999118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fd6e7-168f-48d9-88a6-9988fa568415 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:00.999744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fd6e7-168f-48d9-88a6-9988fa568415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:01.001635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fd6e7-168f-48d9-88a6-9988fa568415 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:01.027944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1270/2516] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:11:00 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:11:01.032200 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3445d1-4215-4c35-ac37-fd0386f557e9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:01.045719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3445d1-4215-4c35-ac37-fd0386f557e9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:01.084457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:01.111766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273012cc-97db-4c76-a0de-facefea78436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:01.112305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273012cc-97db-4c76-a0de-facefea78436 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.112585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273012cc-97db-4c76-a0de-facefea78436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.112858 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273012cc-97db-4c76-a0de-facefea78436 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.113521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273012cc-97db-4c76-a0de-facefea78436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:01.116197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273012cc-97db-4c76-a0de-facefea78436 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:01.136712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1247/2517] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:11:01 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:11:01.140697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fd6e7-168f-48d9-88a6-9988fa568415 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:01.151070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fd6e7-168f-48d9-88a6-9988fa568415 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:01.155804 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fada07c-49b1-48bc-be98-872aecb6bc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:01.156471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada07c-49b1-48bc-be98-872aecb6bc32 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.156753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada07c-49b1-48bc-be98-872aecb6bc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.157021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada07c-49b1-48bc-be98-872aecb6bc32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.157658 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fada07c-49b1-48bc-be98-872aecb6bc32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:01.159269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fada07c-49b1-48bc-be98-872aecb6bc32 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:01.170941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1271/2518] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:11:01.175229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273012cc-97db-4c76-a0de-facefea78436 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:01.186254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273012cc-97db-4c76-a0de-facefea78436 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:01.191302 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cef1ee-f4eb-4f3d-9e3b-d64c5815950a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:01.191902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cef1ee-f4eb-4f3d-9e3b-d64c5815950a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.192183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cef1ee-f4eb-4f3d-9e3b-d64c5815950a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.192495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cef1ee-f4eb-4f3d-9e3b-d64c5815950a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.193136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cef1ee-f4eb-4f3d-9e3b-d64c5815950a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:01.194761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cef1ee-f4eb-4f3d-9e3b-d64c5815950a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:01.205475 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1248/2519] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:11:01.209746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fada07c-49b1-48bc-be98-872aecb6bc32 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:01.219733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fada07c-49b1-48bc-be98-872aecb6bc32 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:01.225239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484132c-a407-47f3-8a8a-a5dd38786254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:01.225889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484132c-a407-47f3-8a8a-a5dd38786254 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.226170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484132c-a407-47f3-8a8a-a5dd38786254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.226478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484132c-a407-47f3-8a8a-a5dd38786254 admin admin] PATH_INFO: `/v3/projects/49a83fe948d94d94aadba849919924f3/users/692a0828d7e145c7b05c8dced615371b/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.241938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484132c-a407-47f3-8a8a-a5dd38786254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a83fe948d94d94aadba849919924f3, user_id=692a0828d7e145c7b05c8dced615371b, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:01.246943 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484132c-a407-47f3-8a8a-a5dd38786254 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:01.291801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1272/2520] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:11:01 2022] PUT /identity/v3/projects/49a83fe948d94d94aadba849919924f3/users/692a0828d7e145c7b05c8dced615371b/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:01.302983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd357419-9e30-4e4c-bad5-47117fb3cd7c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.303778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd357419-9e30-4e4c-bad5-47117fb3cd7c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.303778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd357419-9e30-4e4c-bad5-47117fb3cd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.332501 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bd357419-9e30-4e4c-bad5-47117fb3cd7c None None] MFA Rules not processed for user `692a0828d7e145c7b05c8dced615371b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:01.333706 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bd357419-9e30-4e4c-bad5-47117fb3cd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:01.397739 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1249/2521] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:01.402291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484132c-a407-47f3-8a8a-a5dd38786254 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:01.465618 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484132c-a407-47f3-8a8a-a5dd38786254 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:01.470361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c9fa0-dae0-4d73-b87a-aab49f781340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:01.470963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c9fa0-dae0-4d73-b87a-aab49f781340 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.471283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c9fa0-dae0-4d73-b87a-aab49f781340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.471558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c9fa0-dae0-4d73-b87a-aab49f781340 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.472199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c9fa0-dae0-4d73-b87a-aab49f781340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:01.474660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c9fa0-dae0-4d73-b87a-aab49f781340 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:01.494340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:01.502805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1273/2522] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:11:01 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:11:01.507039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd357419-9e30-4e4c-bad5-47117fb3cd7c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:01.517569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd357419-9e30-4e4c-bad5-47117fb3cd7c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:01.573775 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da422249-a255-445b-8b99-69dc32351117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:01.574385 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da422249-a255-445b-8b99-69dc32351117 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.574701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da422249-a255-445b-8b99-69dc32351117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.574973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da422249-a255-445b-8b99-69dc32351117 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.575617 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da422249-a255-445b-8b99-69dc32351117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:01.577553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da422249-a255-445b-8b99-69dc32351117 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:01.602350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1250/2523] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:11:01 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:11:01.603674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c9fa0-dae0-4d73-b87a-aab49f781340 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:01.614162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c9fa0-dae0-4d73-b87a-aab49f781340 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:01.618881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90183b2-b8b9-4e16-956c-226a27291601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:01.619511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90183b2-b8b9-4e16-956c-226a27291601 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.619792 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90183b2-b8b9-4e16-956c-226a27291601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.620064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90183b2-b8b9-4e16-956c-226a27291601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.620696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90183b2-b8b9-4e16-956c-226a27291601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:01.622271 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90183b2-b8b9-4e16-956c-226a27291601 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:01.632772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1274/2524] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:11:01.636916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da422249-a255-445b-8b99-69dc32351117 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:01.648602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da422249-a255-445b-8b99-69dc32351117 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:01.654696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399690e-2ead-4f52-9d7c-811893a23a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:01.655423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399690e-2ead-4f52-9d7c-811893a23a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.655699 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399690e-2ead-4f52-9d7c-811893a23a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.655967 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399690e-2ead-4f52-9d7c-811893a23a2b admin admin] PATH_INFO: `/v3/projects/a522b99ee4dd456092f3e453315c185f/users/52868efa60084d7187922ea5fbb20a79/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.671882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399690e-2ead-4f52-9d7c-811893a23a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a522b99ee4dd456092f3e453315c185f, user_id=52868efa60084d7187922ea5fbb20a79, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:01.674663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399690e-2ead-4f52-9d7c-811893a23a2b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:01.688360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1251/2525] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:11:01 2022] PUT /identity/v3/projects/a522b99ee4dd456092f3e453315c185f/users/52868efa60084d7187922ea5fbb20a79/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:01.692170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90183b2-b8b9-4e16-956c-226a27291601 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:01.702492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90183b2-b8b9-4e16-956c-226a27291601 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:01.762902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca242840-b117-47fd-bd39-8ed94f159487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:01.763542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca242840-b117-47fd-bd39-8ed94f159487 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.763824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca242840-b117-47fd-bd39-8ed94f159487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.764115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca242840-b117-47fd-bd39-8ed94f159487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.764746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca242840-b117-47fd-bd39-8ed94f159487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:01.766323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca242840-b117-47fd-bd39-8ed94f159487 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:01.776850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1275/2526] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:11:01.781067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399690e-2ead-4f52-9d7c-811893a23a2b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:01.791941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399690e-2ead-4f52-9d7c-811893a23a2b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:01.796534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc9d61-a25d-4793-b354-b16960295d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:01.797171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc9d61-a25d-4793-b354-b16960295d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.797450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc9d61-a25d-4793-b354-b16960295d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.797718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc9d61-a25d-4793-b354-b16960295d35 admin admin] PATH_INFO: `/v3/projects/a522b99ee4dd456092f3e453315c185f/users/52868efa60084d7187922ea5fbb20a79/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.799663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc9d61-a25d-4793-b354-b16960295d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a522b99ee4dd456092f3e453315c185f, user_id=52868efa60084d7187922ea5fbb20a79, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:01.802371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc9d61-a25d-4793-b354-b16960295d35 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:01.805450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:01.813678 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1252/2527] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:11:01 2022] PUT /identity/v3/projects/a522b99ee4dd456092f3e453315c185f/users/52868efa60084d7187922ea5fbb20a79/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:11:01.825165 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798631e-326a-4f28-9087-0c1821a7cff7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:01.825451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798631e-326a-4f28-9087-0c1821a7cff7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:01.825724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798631e-326a-4f28-9087-0c1821a7cff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:01.854201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8798631e-326a-4f28-9087-0c1821a7cff7 None None] MFA Rules not processed for user `52868efa60084d7187922ea5fbb20a79`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:01.855495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8798631e-326a-4f28-9087-0c1821a7cff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:01.922635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1276/2528] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:01.934188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc9d61-a25d-4793-b354-b16960295d35 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:01.935875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c2fc9d61-a25d-4793-b354-b16960295d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:02.001970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc9d61-a25d-4793-b354-b16960295d35 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:02.006350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a35092-1005-4f67-9734-ef9d238a5440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:02.006959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a35092-1005-4f67-9734-ef9d238a5440 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:02.007269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a35092-1005-4f67-9734-ef9d238a5440 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:02.007535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a35092-1005-4f67-9734-ef9d238a5440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:02.064843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a35092-1005-4f67-9734-ef9d238a5440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:02.066275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a35092-1005-4f67-9734-ef9d238a5440 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:02.079008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1253/2529] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:03.605948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798631e-326a-4f28-9087-0c1821a7cff7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:03.607412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8798631e-326a-4f28-9087-0c1821a7cff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:03.619917 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798631e-326a-4f28-9087-0c1821a7cff7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:03.623439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dc938-6804-4db0-bda3-65b5fdbd7000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:03.624030 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc938-6804-4db0-bda3-65b5fdbd7000 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:03.624303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc938-6804-4db0-bda3-65b5fdbd7000 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:03.624569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc938-6804-4db0-bda3-65b5fdbd7000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:03.687851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dc938-6804-4db0-bda3-65b5fdbd7000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:03.689210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dc938-6804-4db0-bda3-65b5fdbd7000 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:03.701510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1277/2530] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:04.236817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a35092-1005-4f67-9734-ef9d238a5440 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:04.245919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a35092-1005-4f67-9734-ef9d238a5440 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:04.249495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2d66a-f29c-41ad-a869-947238cd6edb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:04.250089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2d66a-f29c-41ad-a869-947238cd6edb service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:04.250370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2d66a-f29c-41ad-a869-947238cd6edb service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:04.250638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2d66a-f29c-41ad-a869-947238cd6edb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:04.259413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2d66a-f29c-41ad-a869-947238cd6edb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:04.260826 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2d66a-f29c-41ad-a869-947238cd6edb service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:04.270134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1254/2531] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:09.176076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dc938-6804-4db0-bda3-65b5fdbd7000 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:09.185372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:09.238708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dc938-6804-4db0-bda3-65b5fdbd7000 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:09.243733 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:09.244348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:09.244624 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:09.244889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] PATH_INFO: `/v3/users/d52349a5b2d84daeacad5003726019b5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:09.245981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52349a5b2d84daeacad5003726019b5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:09.247922 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:09.294281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52349a5b2d84daeacad5003726019b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:09.294923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52349a5b2d84daeacad5003726019b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:09.307263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52349a5b2d84daeacad5003726019b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:09.315502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52349a5b2d84daeacad5003726019b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:09.323812 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52349a5b2d84daeacad5003726019b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:09.329853 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1278/2532] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:11:09 2022] DELETE /identity/v3/users/d52349a5b2d84daeacad5003726019b5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:09.334462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f7b68-55fc-41a8-8569-96ca20c6f1a4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:09.334940 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f7b68-55fc-41a8-8569-96ca20c6f1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:09.335102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f7b68-55fc-41a8-8569-96ca20c6f1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:09.353997 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b09f7b68-55fc-41a8-8569-96ca20c6f1a4 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:09.355231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b09f7b68-55fc-41a8-8569-96ca20c6f1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:09.413750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1255/2533] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:09.423353 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:09.424891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:09.483995 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53701755-89f0-4cc8-b4c1-dcf8c1d79ae9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:09.487809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c66acc-abad-4ef2-bab5-11a4381aa7fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:09.488575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66acc-abad-4ef2-bab5-11a4381aa7fc service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:09.488983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66acc-abad-4ef2-bab5-11a4381aa7fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:09.489363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66acc-abad-4ef2-bab5-11a4381aa7fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:09.499007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c66acc-abad-4ef2-bab5-11a4381aa7fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:09.500464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c66acc-abad-4ef2-bab5-11a4381aa7fc service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:09.509763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1279/2534] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:09.783941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f7b68-55fc-41a8-8569-96ca20c6f1a4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:09.785629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b09f7b68-55fc-41a8-8569-96ca20c6f1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:09.798419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f7b68-55fc-41a8-8569-96ca20c6f1a4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:09.810353 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:09.811060 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:09.811358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:09.811607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] PATH_INFO: `/v3/projects/7fd9313005a1492dacc7b73f7b09c179` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:09.812731 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fd9313005a1492dacc7b73f7b09c179)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:09.814756 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:09.850703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:09.863602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fd9313005a1492dacc7b73f7b09c179', 'request_id': 'req-2e158656-b826-493a-ab80-b3262bfbdd35'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:09.870530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fd9313005a1492dacc7b73f7b09c179', 'request_id': 'req-2e158656-b826-493a-ab80-b3262bfbdd35'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:09.876043 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fd9313005a1492dacc7b73f7b09c179', 'request_id': 'req-2e158656-b826-493a-ab80-b3262bfbdd35'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:09.889058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] The token cache is being invalidate because project 7fd9313005a1492dacc7b73f7b09c179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:11:09.889344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:09.889965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:09.891451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1256/2535] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:11:09 2022] DELETE /identity/v3/projects/7fd9313005a1492dacc7b73f7b09c179 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:09.895435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c66acc-abad-4ef2-bab5-11a4381aa7fc service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:09.905246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c66acc-abad-4ef2-bab5-11a4381aa7fc service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:09.931282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:09.963572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:09.964203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:09.964486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:09.964760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] PATH_INFO: `/v3/users/692a0828d7e145c7b05c8dced615371b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:09.965893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] RBAC: Authorizing `identity:delete_user(user_id=692a0828d7e145c7b05c8dced615371b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:09.967850 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:10.015152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '692a0828d7e145c7b05c8dced615371b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.015811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '692a0828d7e145c7b05c8dced615371b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.028208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '692a0828d7e145c7b05c8dced615371b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.035884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '692a0828d7e145c7b05c8dced615371b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.043349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '692a0828d7e145c7b05c8dced615371b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.049466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1280/2536] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:11:09 2022] DELETE /identity/v3/users/692a0828d7e145c7b05c8dced615371b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:10.331849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:10.333316 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:10.346274 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158656-b826-493a-ab80-b3262bfbdd35 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:10.408921 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:10.409605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:10.409880 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:10.410148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] PATH_INFO: `/v3/projects/49a83fe948d94d94aadba849919924f3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:10.411812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] RBAC: Authorizing `identity:delete_project(project_id=49a83fe948d94d94aadba849919924f3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:10.413704 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:10.461244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49a83fe948d94d94aadba849919924f3', 'request_id': 'req-63a96bf7-0470-4111-ae8d-cd49856a863c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.467524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49a83fe948d94d94aadba849919924f3', 'request_id': 'req-63a96bf7-0470-4111-ae8d-cd49856a863c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.472954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49a83fe948d94d94aadba849919924f3', 'request_id': 'req-63a96bf7-0470-4111-ae8d-cd49856a863c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.486406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] The token cache is being invalidate because project 49a83fe948d94d94aadba849919924f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:11:10.486715 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.487339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.488854 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1257/2537] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:11:10 2022] DELETE /identity/v3/projects/49a83fe948d94d94aadba849919924f3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:11:10.493117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:10.494500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:10.507467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269101d8-9a51-4650-9dc5-16cf2c14f573 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:10.564597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d6591-5996-487d-ae8c-b4e0340b30c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:10.565228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d6591-5996-487d-ae8c-b4e0340b30c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:10.565506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d6591-5996-487d-ae8c-b4e0340b30c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:10.565779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d6591-5996-487d-ae8c-b4e0340b30c0 admin admin] PATH_INFO: `/v3/users/52868efa60084d7187922ea5fbb20a79` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:10.566890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d6591-5996-487d-ae8c-b4e0340b30c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52868efa60084d7187922ea5fbb20a79)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:10.568940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d6591-5996-487d-ae8c-b4e0340b30c0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:10.616435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bd4d6591-5996-487d-ae8c-b4e0340b30c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52868efa60084d7187922ea5fbb20a79'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.617076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bd4d6591-5996-487d-ae8c-b4e0340b30c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52868efa60084d7187922ea5fbb20a79'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.618937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:10.630981 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bd4d6591-5996-487d-ae8c-b4e0340b30c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52868efa60084d7187922ea5fbb20a79'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.638034 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bd4d6591-5996-487d-ae8c-b4e0340b30c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52868efa60084d7187922ea5fbb20a79'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.644963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bd4d6591-5996-487d-ae8c-b4e0340b30c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52868efa60084d7187922ea5fbb20a79'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:10.651344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1281/2538] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:11:10 2022] DELETE /identity/v3/users/52868efa60084d7187922ea5fbb20a79 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:10.933494 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:10.935027 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:10.951569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a96bf7-0470-4111-ae8d-cd49856a863c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:11.000148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:11.011994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:11.012628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.012934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.013206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] PATH_INFO: `/v3/projects/a522b99ee4dd456092f3e453315c185f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.014324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a522b99ee4dd456092f3e453315c185f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:11.016292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:11.065777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a522b99ee4dd456092f3e453315c185f', 'request_id': 'req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:11.072691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a522b99ee4dd456092f3e453315c185f', 'request_id': 'req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:11.078464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a522b99ee4dd456092f3e453315c185f', 'request_id': 'req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:11.091499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] The token cache is being invalidate because project a522b99ee4dd456092f3e453315c185f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:11:11.091792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:11.092410 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c46e29d0-33c2-4560-9090-b4a6eb6bf2d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:11.093896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1258/2539] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:11:10 2022] DELETE /identity/v3/projects/a522b99ee4dd456092f3e453315c185f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:11.099925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891ff60-b8b0-4897-ae58-97a1a26ba09f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.100213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891ff60-b8b0-4897-ae58-97a1a26ba09f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.100536 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891ff60-b8b0-4897-ae58-97a1a26ba09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.119699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d891ff60-b8b0-4897-ae58-97a1a26ba09f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:11.120897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d891ff60-b8b0-4897-ae58-97a1a26ba09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:11.187073 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1282/2540] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:11.195619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e236b-8827-42ac-b047-046a56d51c68 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.195934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e236b-8827-42ac-b047-046a56d51c68 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.196214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e236b-8827-42ac-b047-046a56d51c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.217757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-dc6e236b-8827-42ac-b047-046a56d51c68 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:11.218981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dc6e236b-8827-42ac-b047-046a56d51c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:11.227282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1259/2541] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:11.231493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891ff60-b8b0-4897-ae58-97a1a26ba09f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:11.241759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891ff60-b8b0-4897-ae58-97a1a26ba09f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:11.246343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dacc51-4c1e-4ec4-a317-f88277d75c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:11.246931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dacc51-4c1e-4ec4-a317-f88277d75c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.247240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dacc51-4c1e-4ec4-a317-f88277d75c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.247513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dacc51-4c1e-4ec4-a317-f88277d75c64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.248153 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dacc51-4c1e-4ec4-a317-f88277d75c64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:11:11.248455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dacc51-4c1e-4ec4-a317-f88277d75c64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:11.250040 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dacc51-4c1e-4ec4-a317-f88277d75c64 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:11.261941 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1283/2542] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:11:11.266245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e236b-8827-42ac-b047-046a56d51c68 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:11.276289 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e236b-8827-42ac-b047-046a56d51c68 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:11.281187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3382cf6-50c2-4c9c-be87-924a630ed8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:11.281819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3382cf6-50c2-4c9c-be87-924a630ed8db admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.282108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3382cf6-50c2-4c9c-be87-924a630ed8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.282407 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3382cf6-50c2-4c9c-be87-924a630ed8db admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.283131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3382cf6-50c2-4c9c-be87-924a630ed8db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:11.285122 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3382cf6-50c2-4c9c-be87-924a630ed8db admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:11.312815 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1260/2543] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:11:11 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:11:11.317087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dacc51-4c1e-4ec4-a317-f88277d75c64 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:11.327903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dacc51-4c1e-4ec4-a317-f88277d75c64 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:11.385399 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd435205-63ff-4519-98ca-2d6ed8093207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:11.386013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd435205-63ff-4519-98ca-2d6ed8093207 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.386296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd435205-63ff-4519-98ca-2d6ed8093207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.386725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd435205-63ff-4519-98ca-2d6ed8093207 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.387407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd435205-63ff-4519-98ca-2d6ed8093207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:11.390285 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd435205-63ff-4519-98ca-2d6ed8093207 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:11.411323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1284/2544] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:11:11 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:11:11.415416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3382cf6-50c2-4c9c-be87-924a630ed8db admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:11.425531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3382cf6-50c2-4c9c-be87-924a630ed8db admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:11.430568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b1214-eb11-4648-b23d-b787963e9f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:11.431182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b1214-eb11-4648-b23d-b787963e9f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.431461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b1214-eb11-4648-b23d-b787963e9f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.431729 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b1214-eb11-4648-b23d-b787963e9f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.432355 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b1214-eb11-4648-b23d-b787963e9f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:11.433933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b1214-eb11-4648-b23d-b787963e9f5c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:11.444383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1261/2545] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:11:11.448428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd435205-63ff-4519-98ca-2d6ed8093207 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:11.458873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd435205-63ff-4519-98ca-2d6ed8093207 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:11.463586 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda820c-f4ee-44a4-bccc-669f1fc69b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:11.464236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda820c-f4ee-44a4-bccc-669f1fc69b12 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.464552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda820c-f4ee-44a4-bccc-669f1fc69b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.464845 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda820c-f4ee-44a4-bccc-669f1fc69b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.465479 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda820c-f4ee-44a4-bccc-669f1fc69b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:11.467201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda820c-f4ee-44a4-bccc-669f1fc69b12 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:11.478123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1285/2546] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:11:11.482278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b1214-eb11-4648-b23d-b787963e9f5c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:11.492930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b1214-eb11-4648-b23d-b787963e9f5c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:11.497574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6cb3b-94ba-4256-b2a4-23ca3d8fe1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:11.498214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6cb3b-94ba-4256-b2a4-23ca3d8fe1b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.498490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6cb3b-94ba-4256-b2a4-23ca3d8fe1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.498759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6cb3b-94ba-4256-b2a4-23ca3d8fe1b6 admin admin] PATH_INFO: `/v3/projects/b62ef6fe6e67488fbfee2785329b3225/users/72e9f550281e4db782a06df404c3c8a0/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.513706 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6cb3b-94ba-4256-b2a4-23ca3d8fe1b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62ef6fe6e67488fbfee2785329b3225, user_id=72e9f550281e4db782a06df404c3c8a0, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:11.516382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6cb3b-94ba-4256-b2a4-23ca3d8fe1b6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:11.529102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1262/2547] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:11:11 2022] PUT /identity/v3/projects/b62ef6fe6e67488fbfee2785329b3225/users/72e9f550281e4db782a06df404c3c8a0/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:11.540118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31437e38-c3ae-49fc-9e02-89c1adfae10d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.540611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31437e38-c3ae-49fc-9e02-89c1adfae10d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.540724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31437e38-c3ae-49fc-9e02-89c1adfae10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.569821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-31437e38-c3ae-49fc-9e02-89c1adfae10d None None] MFA Rules not processed for user `72e9f550281e4db782a06df404c3c8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:11.571116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-31437e38-c3ae-49fc-9e02-89c1adfae10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:11.590843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:11.636309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1286/2548] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:11.648145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6cb3b-94ba-4256-b2a4-23ca3d8fe1b6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:11.649786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-08d6cb3b-94ba-4256-b2a4-23ca3d8fe1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:11.708945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6cb3b-94ba-4256-b2a4-23ca3d8fe1b6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:11.712893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f8a99-0deb-467e-8236-810623718b9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:11.713500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8a99-0deb-467e-8236-810623718b9c service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.713902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8a99-0deb-467e-8236-810623718b9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.714179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8a99-0deb-467e-8236-810623718b9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.723936 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f8a99-0deb-467e-8236-810623718b9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:11.725393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f8a99-0deb-467e-8236-810623718b9c service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:11.734140 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1263/2549] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:11.758864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31437e38-c3ae-49fc-9e02-89c1adfae10d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:11.821485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31437e38-c3ae-49fc-9e02-89c1adfae10d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:11.826271 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:11.826885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.827298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.827575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] PATH_INFO: `/v3/users/72e9f550281e4db782a06df404c3c8a0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.828779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=72e9f550281e4db782a06df404c3c8a0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:11.830756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:11.876323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72e9f550281e4db782a06df404c3c8a0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:11.877079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72e9f550281e4db782a06df404c3c8a0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:11.890132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72e9f550281e4db782a06df404c3c8a0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:11.897697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72e9f550281e4db782a06df404c3c8a0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:11.904701 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72e9f550281e4db782a06df404c3c8a0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:11.910683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1287/2550] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:11:11 2022] DELETE /identity/v3/users/72e9f550281e4db782a06df404c3c8a0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:11.915296 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e4fad-3df4-4808-b09d-85ba699f720a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:11.915584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e4fad-3df4-4808-b09d-85ba699f720a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:11.915860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e4fad-3df4-4808-b09d-85ba699f720a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:11.934556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c92e4fad-3df4-4808-b09d-85ba699f720a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:11.935791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c92e4fad-3df4-4808-b09d-85ba699f720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:11.948945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:11.995430 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1264/2551] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:11:12.005686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:12.007124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:12.051515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:12.064906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea8002-b864-431c-9f6f-59dd1bd2eb0d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:12.068818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f3b41-fd4b-448e-bde2-8ac38d297c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:12.069414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f3b41-fd4b-448e-bde2-8ac38d297c8d service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:12.069689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f3b41-fd4b-448e-bde2-8ac38d297c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:12.069955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f3b41-fd4b-448e-bde2-8ac38d297c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:12.079720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f3b41-fd4b-448e-bde2-8ac38d297c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:12.081070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f3b41-fd4b-448e-bde2-8ac38d297c8d service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:12.090320 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1288/2552] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:11:12.373480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e4fad-3df4-4808-b09d-85ba699f720a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:12.374973 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c92e4fad-3df4-4808-b09d-85ba699f720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:12.388252 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e4fad-3df4-4808-b09d-85ba699f720a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:12.393438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:12.394059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:12.394362 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:12.394635 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] PATH_INFO: `/v3/projects/b62ef6fe6e67488fbfee2785329b3225` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:12.395797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62ef6fe6e67488fbfee2785329b3225)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:12.397726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:12.452949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62ef6fe6e67488fbfee2785329b3225', 'request_id': 'req-041b382f-d36d-456f-97fc-357af574b7f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:12.459695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62ef6fe6e67488fbfee2785329b3225', 'request_id': 'req-041b382f-d36d-456f-97fc-357af574b7f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:12.466781 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62ef6fe6e67488fbfee2785329b3225', 'request_id': 'req-041b382f-d36d-456f-97fc-357af574b7f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:12.481067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] The token cache is being invalidate because project b62ef6fe6e67488fbfee2785329b3225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:11:12.481374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:12.482019 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-041b382f-d36d-456f-97fc-357af574b7f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:12.483629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1265/2553] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:11:12 2022] DELETE /identity/v3/projects/b62ef6fe6e67488fbfee2785329b3225 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:11:12.488537 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d1225-18f2-4cb9-9d8e-152d605f8088 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:12.488862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d1225-18f2-4cb9-9d8e-152d605f8088 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:12.489168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d1225-18f2-4cb9-9d8e-152d605f8088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:12.509247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-059d1225-18f2-4cb9-9d8e-152d605f8088 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:12.510448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-059d1225-18f2-4cb9-9d8e-152d605f8088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:12.568765 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1289/2554] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:12.576705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7ca38-4c9e-4697-9909-ea2e4cad0064 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:12.576991 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7ca38-4c9e-4697-9909-ea2e4cad0064 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:12.577295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7ca38-4c9e-4697-9909-ea2e4cad0064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:12.596860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-adc7ca38-4c9e-4697-9909-ea2e4cad0064 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:12.598094 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-adc7ca38-4c9e-4697-9909-ea2e4cad0064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:12.606841 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1266/2555] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:12.611026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d1225-18f2-4cb9-9d8e-152d605f8088 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:12.621349 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d1225-18f2-4cb9-9d8e-152d605f8088 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:12.625831 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582baa1-c46b-49fc-baa5-261f8ba486f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:12.626498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582baa1-c46b-49fc-baa5-261f8ba486f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:12.626781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582baa1-c46b-49fc-baa5-261f8ba486f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:12.627085 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582baa1-c46b-49fc-baa5-261f8ba486f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:12.627734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582baa1-c46b-49fc-baa5-261f8ba486f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:11:12.628040 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582baa1-c46b-49fc-baa5-261f8ba486f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:12.629619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582baa1-c46b-49fc-baa5-261f8ba486f0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:12.641140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1290/2556] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:11:12.645269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7ca38-4c9e-4697-9909-ea2e4cad0064 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:12.654569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7ca38-4c9e-4697-9909-ea2e4cad0064 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:12.659288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622a320-bd89-4cf4-8e89-10be4691822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:12.659881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622a320-bd89-4cf4-8e89-10be4691822b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:12.660162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622a320-bd89-4cf4-8e89-10be4691822b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:12.660438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622a320-bd89-4cf4-8e89-10be4691822b admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:12.661072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622a320-bd89-4cf4-8e89-10be4691822b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:12.663510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622a320-bd89-4cf4-8e89-10be4691822b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:12.698664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1267/2557] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:11:12 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:11:12.699883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582baa1-c46b-49fc-baa5-261f8ba486f0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:12.710406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582baa1-c46b-49fc-baa5-261f8ba486f0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:12.784244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f31f7-a2cd-4b03-848e-57d919609a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:12.784848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f31f7-a2cd-4b03-848e-57d919609a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:12.785123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f31f7-a2cd-4b03-848e-57d919609a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:12.785390 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f31f7-a2cd-4b03-848e-57d919609a45 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:12.786012 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f31f7-a2cd-4b03-848e-57d919609a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:12.788551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f31f7-a2cd-4b03-848e-57d919609a45 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:12.808971 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1291/2558] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:11:12 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:11:12.813285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622a320-bd89-4cf4-8e89-10be4691822b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:12.823997 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622a320-bd89-4cf4-8e89-10be4691822b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:12.828477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f2d1d-8641-47b8-9c46-e99617de26a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:12.829062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f2d1d-8641-47b8-9c46-e99617de26a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:12.829350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f2d1d-8641-47b8-9c46-e99617de26a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:12.829610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f2d1d-8641-47b8-9c46-e99617de26a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:12.830379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f2d1d-8641-47b8-9c46-e99617de26a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:12.832472 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f2d1d-8641-47b8-9c46-e99617de26a6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:12.843201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1268/2559] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:11:12.847231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f31f7-a2cd-4b03-848e-57d919609a45 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:12.858212 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f31f7-a2cd-4b03-848e-57d919609a45 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:12.862821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2188fdc-bb37-486b-82d4-c55c8764250e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:12.863441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2188fdc-bb37-486b-82d4-c55c8764250e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:12.863723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2188fdc-bb37-486b-82d4-c55c8764250e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:12.863994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2188fdc-bb37-486b-82d4-c55c8764250e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:12.865164 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2188fdc-bb37-486b-82d4-c55c8764250e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:12.866737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2188fdc-bb37-486b-82d4-c55c8764250e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:12.877310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1292/2560] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:11:12.881467 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f2d1d-8641-47b8-9c46-e99617de26a6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:12.892186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f2d1d-8641-47b8-9c46-e99617de26a6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:12.896793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21aaa05-caa9-4a1a-a7f5-1771f3d3d2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:12.897428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21aaa05-caa9-4a1a-a7f5-1771f3d3d2c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:12.897727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21aaa05-caa9-4a1a-a7f5-1771f3d3d2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:12.897986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21aaa05-caa9-4a1a-a7f5-1771f3d3d2c5 admin admin] PATH_INFO: `/v3/projects/592f2e7ec5cc4280abb98d05588bd360/users/b3c083daf9804821bbb399dff7e2e680/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:12.912356 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21aaa05-caa9-4a1a-a7f5-1771f3d3d2c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=592f2e7ec5cc4280abb98d05588bd360, user_id=b3c083daf9804821bbb399dff7e2e680, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:12.915018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21aaa05-caa9-4a1a-a7f5-1771f3d3d2c5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:12.928740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1269/2561] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:11:12 2022] PUT /identity/v3/projects/592f2e7ec5cc4280abb98d05588bd360/users/b3c083daf9804821bbb399dff7e2e680/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:12.939218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aab412-406d-4be9-ac47-1a037dc7de56 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:12.939505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aab412-406d-4be9-ac47-1a037dc7de56 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:12.939779 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aab412-406d-4be9-ac47-1a037dc7de56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:12.968000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c6aab412-406d-4be9-ac47-1a037dc7de56 None None] MFA Rules not processed for user `b3c083daf9804821bbb399dff7e2e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:12.969249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c6aab412-406d-4be9-ac47-1a037dc7de56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:13.011669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:13.036986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1293/2562] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:13.047707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21aaa05-caa9-4a1a-a7f5-1771f3d3d2c5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:13.049521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e21aaa05-caa9-4a1a-a7f5-1771f3d3d2c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:13.093939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:13.109534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21aaa05-caa9-4a1a-a7f5-1771f3d3d2c5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:13.113112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a169eb-158b-4a90-821b-4e9521802911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:13.113711 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a169eb-158b-4a90-821b-4e9521802911 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:13.113995 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a169eb-158b-4a90-821b-4e9521802911 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:13.114282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a169eb-158b-4a90-821b-4e9521802911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:13.122873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a169eb-158b-4a90-821b-4e9521802911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:13.124304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a169eb-158b-4a90-821b-4e9521802911 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:13.133111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1270/2563] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:23.023693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aab412-406d-4be9-ac47-1a037dc7de56 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:23.085439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aab412-406d-4be9-ac47-1a037dc7de56 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:23.089918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:23.090540 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:23.090818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:23.091118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] PATH_INFO: `/v3/users/b3c083daf9804821bbb399dff7e2e680` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:23.092227 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c083daf9804821bbb399dff7e2e680)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:23.094130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:23.128910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=8 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:23.139526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c083daf9804821bbb399dff7e2e680'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:23.140171 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c083daf9804821bbb399dff7e2e680'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:23.152517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c083daf9804821bbb399dff7e2e680'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:23.159678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c083daf9804821bbb399dff7e2e680'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:23.166488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c083daf9804821bbb399dff7e2e680'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:23.173286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1294/2564] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:11:23 2022] DELETE /identity/v3/users/b3c083daf9804821bbb399dff7e2e680 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:23.177801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c76e86-a6cb-4673-aec9-dc0fbff406c6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:23.178087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c76e86-a6cb-4673-aec9-dc0fbff406c6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:23.178365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c76e86-a6cb-4673-aec9-dc0fbff406c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:23.197432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-83c76e86-a6cb-4673-aec9-dc0fbff406c6 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:23.198905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-83c76e86-a6cb-4673-aec9-dc0fbff406c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:23.257206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1271/2565] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:23.266898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:23.268344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:23.325052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4db230-f0d2-4d53-83d4-ede7c2e73173 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:23.328671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6bd27-c1fb-421c-afa6-6bc524266f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:23.329408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6bd27-c1fb-421c-afa6-6bc524266f1b service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:23.329684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6bd27-c1fb-421c-afa6-6bc524266f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:23.329952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6bd27-c1fb-421c-afa6-6bc524266f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:23.338706 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6bd27-c1fb-421c-afa6-6bc524266f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:23.340179 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6bd27-c1fb-421c-afa6-6bc524266f1b service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:23.349568 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1295/2566] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:23.611645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c76e86-a6cb-4673-aec9-dc0fbff406c6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:23.613202 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-83c76e86-a6cb-4673-aec9-dc0fbff406c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:23.630247 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c76e86-a6cb-4673-aec9-dc0fbff406c6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:23.635312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:23.635994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:23.636299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:23.636602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] PATH_INFO: `/v3/projects/592f2e7ec5cc4280abb98d05588bd360` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:23.637773 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=592f2e7ec5cc4280abb98d05588bd360)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:23.639808 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:23.696090 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592f2e7ec5cc4280abb98d05588bd360', 'request_id': 'req-ebe2f48f-b178-474b-a9ea-4114c1be2317'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:23.702511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592f2e7ec5cc4280abb98d05588bd360', 'request_id': 'req-ebe2f48f-b178-474b-a9ea-4114c1be2317'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:23.708097 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592f2e7ec5cc4280abb98d05588bd360', 'request_id': 'req-ebe2f48f-b178-474b-a9ea-4114c1be2317'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:23.720238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] The token cache is being invalidate because project 592f2e7ec5cc4280abb98d05588bd360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:11:23.720521 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:23.721128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ebe2f48f-b178-474b-a9ea-4114c1be2317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:11:23.722607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1272/2567] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:11:23 2022] DELETE /identity/v3/projects/592f2e7ec5cc4280abb98d05588bd360 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:11:23.728021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af31671-1eae-485f-8bd0-11e93b4b6f83 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:23.728310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af31671-1eae-485f-8bd0-11e93b4b6f83 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:23.728581 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af31671-1eae-485f-8bd0-11e93b4b6f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:23.736025 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:23.750563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1af31671-1eae-485f-8bd0-11e93b4b6f83 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:23.751811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1af31671-1eae-485f-8bd0-11e93b4b6f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:23.812884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1296/2568] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:23.821286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829d939-6ada-47b6-a443-ca5479c5c6a4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:23.821570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829d939-6ada-47b6-a443-ca5479c5c6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:23.821845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829d939-6ada-47b6-a443-ca5479c5c6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:23.842387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6829d939-6ada-47b6-a443-ca5479c5c6a4 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:23.843619 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6829d939-6ada-47b6-a443-ca5479c5c6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:23.851795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1273/2569] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:23.855939 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af31671-1eae-485f-8bd0-11e93b4b6f83 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:23.865867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af31671-1eae-485f-8bd0-11e93b4b6f83 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:23.870677 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad02f45-d52d-469c-a9c7-1d883c09216e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:23.871445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad02f45-d52d-469c-a9c7-1d883c09216e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:23.871734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad02f45-d52d-469c-a9c7-1d883c09216e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:23.872010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad02f45-d52d-469c-a9c7-1d883c09216e admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:23.872689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad02f45-d52d-469c-a9c7-1d883c09216e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:11:23.872994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad02f45-d52d-469c-a9c7-1d883c09216e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:23.874682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad02f45-d52d-469c-a9c7-1d883c09216e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:23.886844 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1297/2570] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:11:23.890965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829d939-6ada-47b6-a443-ca5479c5c6a4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:23.900800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829d939-6ada-47b6-a443-ca5479c5c6a4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:23.905465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cd1c5-d82f-4429-a190-23f0df6b2643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:23.906054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cd1c5-d82f-4429-a190-23f0df6b2643 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:23.906333 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cd1c5-d82f-4429-a190-23f0df6b2643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:23.906605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cd1c5-d82f-4429-a190-23f0df6b2643 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:23.907251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cd1c5-d82f-4429-a190-23f0df6b2643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:23.909151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cd1c5-d82f-4429-a190-23f0df6b2643 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:23.936324 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1274/2571] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:11:23 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:11:23.940645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad02f45-d52d-469c-a9c7-1d883c09216e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:23.951623 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad02f45-d52d-469c-a9c7-1d883c09216e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:24.009070 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca54f2b-cd9c-48e3-bc21-e64d3ae4232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:24.009667 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca54f2b-cd9c-48e3-bc21-e64d3ae4232f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:24.009945 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca54f2b-cd9c-48e3-bc21-e64d3ae4232f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:24.010215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca54f2b-cd9c-48e3-bc21-e64d3ae4232f admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:24.010829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca54f2b-cd9c-48e3-bc21-e64d3ae4232f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:24.012755 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca54f2b-cd9c-48e3-bc21-e64d3ae4232f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:24.035588 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1298/2572] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:11:23 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:11:24.040575 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cd1c5-d82f-4429-a190-23f0df6b2643 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:24.052752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cd1c5-d82f-4429-a190-23f0df6b2643 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:24.058564 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08557-90a5-4389-bec8-a1b2fb92ce65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:24.059216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08557-90a5-4389-bec8-a1b2fb92ce65 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:24.059502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08557-90a5-4389-bec8-a1b2fb92ce65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:24.059803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08557-90a5-4389-bec8-a1b2fb92ce65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:24.060492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08557-90a5-4389-bec8-a1b2fb92ce65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:24.062145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08557-90a5-4389-bec8-a1b2fb92ce65 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:24.073080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1275/2573] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:11:24.077340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca54f2b-cd9c-48e3-bc21-e64d3ae4232f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:24.087893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca54f2b-cd9c-48e3-bc21-e64d3ae4232f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:24.092720 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c6326-f106-4350-b5c2-3c1b6bab4483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:24.093341 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c6326-f106-4350-b5c2-3c1b6bab4483 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:24.093641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c6326-f106-4350-b5c2-3c1b6bab4483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:24.093935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c6326-f106-4350-b5c2-3c1b6bab4483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:24.094684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c6326-f106-4350-b5c2-3c1b6bab4483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:24.096406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c6326-f106-4350-b5c2-3c1b6bab4483 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:24.107274 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1299/2574] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:11:24.111453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08557-90a5-4389-bec8-a1b2fb92ce65 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:24.122110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08557-90a5-4389-bec8-a1b2fb92ce65 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:24.126946 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099d6fa-46c8-4299-8d0a-e7921d9e91b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:24.127614 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099d6fa-46c8-4299-8d0a-e7921d9e91b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:24.127896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099d6fa-46c8-4299-8d0a-e7921d9e91b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:24.128174 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099d6fa-46c8-4299-8d0a-e7921d9e91b5 admin admin] PATH_INFO: `/v3/projects/3d7f020293da4a45a541d6ca1397380d/users/260ffd33f3c443c09fdb8d27a1de846e/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:24.143390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099d6fa-46c8-4299-8d0a-e7921d9e91b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7f020293da4a45a541d6ca1397380d, user_id=260ffd33f3c443c09fdb8d27a1de846e, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:24.146301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099d6fa-46c8-4299-8d0a-e7921d9e91b5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:24.159418 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1276/2575] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:11:24 2022] PUT /identity/v3/projects/3d7f020293da4a45a541d6ca1397380d/users/260ffd33f3c443c09fdb8d27a1de846e/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:24.164486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489c2ea-24c4-4310-9932-4dfee659b313 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:24.164772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489c2ea-24c4-4310-9932-4dfee659b313 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:24.165046 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489c2ea-24c4-4310-9932-4dfee659b313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:24.185084 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b489c2ea-24c4-4310-9932-4dfee659b313 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:24.186279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b489c2ea-24c4-4310-9932-4dfee659b313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:24.250145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1300/2576] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:24.260756 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099d6fa-46c8-4299-8d0a-e7921d9e91b5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:24.262502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8099d6fa-46c8-4299-8d0a-e7921d9e91b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:24.283824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:24.323850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099d6fa-46c8-4299-8d0a-e7921d9e91b5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:24.327597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79d8c7-ca3d-4cdd-a069-6f5d5dc6b318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:24.328219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79d8c7-ca3d-4cdd-a069-6f5d5dc6b318 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:24.328485 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79d8c7-ca3d-4cdd-a069-6f5d5dc6b318 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:24.328756 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79d8c7-ca3d-4cdd-a069-6f5d5dc6b318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:24.338123 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79d8c7-ca3d-4cdd-a069-6f5d5dc6b318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:24.339583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79d8c7-ca3d-4cdd-a069-6f5d5dc6b318 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:24.349461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1277/2577] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:28.393613 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11da69f-e1db-4274-a477-7b0f628cd948 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:28.394108 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11da69f-e1db-4274-a477-7b0f628cd948 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:28.394231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11da69f-e1db-4274-a477-7b0f628cd948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:28.416583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:28.430423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a11da69f-e1db-4274-a477-7b0f628cd948 None None] MFA Rules not processed for user `260ffd33f3c443c09fdb8d27a1de846e`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:28.432007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a11da69f-e1db-4274-a477-7b0f628cd948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:28.517879 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1301/2578] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:28.522834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79d8c7-ca3d-4cdd-a069-6f5d5dc6b318 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:28.534319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79d8c7-ca3d-4cdd-a069-6f5d5dc6b318 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:28.540696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fe230-c511-4fec-8a75-66817a033c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:28.542251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe230-c511-4fec-8a75-66817a033c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:28.542715 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe230-c511-4fec-8a75-66817a033c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:28.543051 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe230-c511-4fec-8a75-66817a033c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:28.543672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fe230-c511-4fec-8a75-66817a033c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:28.545574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fe230-c511-4fec-8a75-66817a033c0f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:28.580629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1278/2579] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:11:28 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:11:28.581738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11da69f-e1db-4274-a477-7b0f628cd948 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:28.607890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11da69f-e1db-4274-a477-7b0f628cd948 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:28.692099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8182f-2fd6-4c95-81c7-0e883581ac15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:28.693101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8182f-2fd6-4c95-81c7-0e883581ac15 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:28.693101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8182f-2fd6-4c95-81c7-0e883581ac15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:28.693276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8182f-2fd6-4c95-81c7-0e883581ac15 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:28.693822 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8182f-2fd6-4c95-81c7-0e883581ac15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:28.695846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8182f-2fd6-4c95-81c7-0e883581ac15 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:28.721633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1302/2580] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:11:28 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:11:28.726657 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fe230-c511-4fec-8a75-66817a033c0f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:28.739644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fe230-c511-4fec-8a75-66817a033c0f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:28.745707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47041593-be77-4d49-8d13-317456978cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:28.746664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47041593-be77-4d49-8d13-317456978cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:28.747076 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47041593-be77-4d49-8d13-317456978cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:28.747666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47041593-be77-4d49-8d13-317456978cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:28.750112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47041593-be77-4d49-8d13-317456978cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:28.752399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47041593-be77-4d49-8d13-317456978cf7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:28.767048 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1279/2581] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:11:28.769581 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8182f-2fd6-4c95-81c7-0e883581ac15 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:28.781163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8182f-2fd6-4c95-81c7-0e883581ac15 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:28.787957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61c661-f465-4768-be7d-19229e93e52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:28.789045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61c661-f465-4768-be7d-19229e93e52a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:28.789580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61c661-f465-4768-be7d-19229e93e52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:28.790162 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61c661-f465-4768-be7d-19229e93e52a admin admin] PATH_INFO: `/v3/projects/94fcc042687d4af0b8d8428c56bb60ef/users/a96c7820b4724e8eb0becb5d736445a9/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:28.810183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61c661-f465-4768-be7d-19229e93e52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fcc042687d4af0b8d8428c56bb60ef, user_id=a96c7820b4724e8eb0becb5d736445a9, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:28.813213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61c661-f465-4768-be7d-19229e93e52a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:28.830463 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1303/2582] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:11:28 2022] PUT /identity/v3/projects/94fcc042687d4af0b8d8428c56bb60ef/users/a96c7820b4724e8eb0becb5d736445a9/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:28.834508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47041593-be77-4d49-8d13-317456978cf7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:28.850176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47041593-be77-4d49-8d13-317456978cf7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:28.919214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55f054-d1d3-4981-8a5a-f6e90275663c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:28.919214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55f054-d1d3-4981-8a5a-f6e90275663c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:28.919214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55f054-d1d3-4981-8a5a-f6e90275663c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:28.919214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55f054-d1d3-4981-8a5a-f6e90275663c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:28.919653 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55f054-d1d3-4981-8a5a-f6e90275663c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:28.922650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55f054-d1d3-4981-8a5a-f6e90275663c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:28.931228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1280/2583] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:11:28.935049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61c661-f465-4768-be7d-19229e93e52a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:28.946439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61c661-f465-4768-be7d-19229e93e52a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:28.951520 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b769f-b8da-4b38-a831-4cff4be5bb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:28.952186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b769f-b8da-4b38-a831-4cff4be5bb64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:28.952464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b769f-b8da-4b38-a831-4cff4be5bb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:28.952867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b769f-b8da-4b38-a831-4cff4be5bb64 admin admin] PATH_INFO: `/v3/projects/94fcc042687d4af0b8d8428c56bb60ef/users/a96c7820b4724e8eb0becb5d736445a9/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:28.954969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b769f-b8da-4b38-a831-4cff4be5bb64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fcc042687d4af0b8d8428c56bb60ef, user_id=a96c7820b4724e8eb0becb5d736445a9, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:28.957618 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b769f-b8da-4b38-a831-4cff4be5bb64 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:28.969245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1304/2584] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:11:28 2022] PUT /identity/v3/projects/94fcc042687d4af0b8d8428c56bb60ef/users/a96c7820b4724e8eb0becb5d736445a9/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:11:33.750291 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d80c5-41b8-4a76-b31f-97c8452e889e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:33.750709 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d80c5-41b8-4a76-b31f-97c8452e889e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:33.750897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d80c5-41b8-4a76-b31f-97c8452e889e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:33.781354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6b8d80c5-41b8-4a76-b31f-97c8452e889e None None] MFA Rules not processed for user `a96c7820b4724e8eb0becb5d736445a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:33.783031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6b8d80c5-41b8-4a76-b31f-97c8452e889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:33.863912 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1281/2585] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:33.881919 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b769f-b8da-4b38-a831-4cff4be5bb64 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:33.884245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aa2b769f-b8da-4b38-a831-4cff4be5bb64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:33.960027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:33.984772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b769f-b8da-4b38-a831-4cff4be5bb64 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:33.995107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5aae5-f51f-4c4a-a2b0-93505126f83e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:33.995877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5aae5-f51f-4c4a-a2b0-93505126f83e service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:33.996340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5aae5-f51f-4c4a-a2b0-93505126f83e service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:33.996782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5aae5-f51f-4c4a-a2b0-93505126f83e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:34.094064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5aae5-f51f-4c4a-a2b0-93505126f83e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:34.095810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5aae5-f51f-4c4a-a2b0-93505126f83e service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:34.124130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1305/2586] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:35.178893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a02d-dd9a-4a70-81b5-1a28b18b76c1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:35.179413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a02d-dd9a-4a70-81b5-1a28b18b76c1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:35.179528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a02d-dd9a-4a70-81b5-1a28b18b76c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:35.199794 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ec75a02d-dd9a-4a70-81b5-1a28b18b76c1 None None] MFA Rules not processed for user `260ffd33f3c443c09fdb8d27a1de846e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:35.201288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ec75a02d-dd9a-4a70-81b5-1a28b18b76c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:35.209092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1282/2587] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:37.631759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5aae5-f51f-4c4a-a2b0-93505126f83e service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:37.687268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5aae5-f51f-4c4a-a2b0-93505126f83e service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:37.690963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457f54f-f9e7-4f3f-b717-b75798a2646a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:37.691595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457f54f-f9e7-4f3f-b717-b75798a2646a service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:37.692324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457f54f-f9e7-4f3f-b717-b75798a2646a service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:37.692324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457f54f-f9e7-4f3f-b717-b75798a2646a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:37.693951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e457f54f-f9e7-4f3f-b717-b75798a2646a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:37.708456 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457f54f-f9e7-4f3f-b717-b75798a2646a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:37.709888 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457f54f-f9e7-4f3f-b717-b75798a2646a service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:37.722083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1306/2588] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:38.916396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75a02d-dd9a-4a70-81b5-1a28b18b76c1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:38.922402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:38.926812 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75a02d-dd9a-4a70-81b5-1a28b18b76c1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:38.931292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c02a54-9b3e-4b44-989a-ac6332f0c309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:38.931905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c02a54-9b3e-4b44-989a-ac6332f0c309 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:38.932186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c02a54-9b3e-4b44-989a-ac6332f0c309 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:38.932464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c02a54-9b3e-4b44-989a-ac6332f0c309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:38.934199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c4c02a54-9b3e-4b44-989a-ac6332f0c309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:38.948965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c02a54-9b3e-4b44-989a-ac6332f0c309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:38.950393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c02a54-9b3e-4b44-989a-ac6332f0c309 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:38.963198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1283/2589] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:47.717308 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=59 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:48.435644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457f54f-f9e7-4f3f-b717-b75798a2646a service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:48.437119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e457f54f-f9e7-4f3f-b717-b75798a2646a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:48.452313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457f54f-f9e7-4f3f-b717-b75798a2646a service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:48.458846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58674cec-1073-4815-befe-c99942be7436 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:48.459478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58674cec-1073-4815-befe-c99942be7436 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:48.459785 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58674cec-1073-4815-befe-c99942be7436 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:48.460063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58674cec-1073-4815-befe-c99942be7436 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:48.461734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-58674cec-1073-4815-befe-c99942be7436 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:48.542421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58674cec-1073-4815-befe-c99942be7436 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:48.544152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58674cec-1073-4815-befe-c99942be7436 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:48.562795 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1307/2590] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:48.957766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:58.435631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c20cb-1930-44fd-a4f3-8fb1902cd884 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:58.436248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c20cb-1930-44fd-a4f3-8fb1902cd884 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:58.436248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c20cb-1930-44fd-a4f3-8fb1902cd884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:58.491666 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-671c20cb-1930-44fd-a4f3-8fb1902cd884 None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:58.492943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-671c20cb-1930-44fd-a4f3-8fb1902cd884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:58.559762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1284/2591] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:11:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:58.560137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=41 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:58.608659 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58674cec-1073-4815-befe-c99942be7436 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:58.610228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-58674cec-1073-4815-befe-c99942be7436 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:58.679531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58674cec-1073-4815-befe-c99942be7436 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:58.684962 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abd503-948d-4d3b-90b9-91c3d38c9302 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:58.685576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd503-948d-4d3b-90b9-91c3d38c9302 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:58.685854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd503-948d-4d3b-90b9-91c3d38c9302 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:58.686123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd503-948d-4d3b-90b9-91c3d38c9302 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:58.695306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abd503-948d-4d3b-90b9-91c3d38c9302 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:58.696890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abd503-948d-4d3b-90b9-91c3d38c9302 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:58.705150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1308/2592] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:11:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:11:58.828429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02c33-8069-47cc-bb02-8b6f31b3305b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:58.828944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02c33-8069-47cc-bb02-8b6f31b3305b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:58.829433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02c33-8069-47cc-bb02-8b6f31b3305b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:58.860822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-94f02c33-8069-47cc-bb02-8b6f31b3305b None None] MFA Rules not processed for user `0547f15d00f043b99de10f3db7348c62`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:58.861978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-94f02c33-8069-47cc-bb02-8b6f31b3305b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:58.940819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1285/2593] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:11:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:59.873868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abd503-948d-4d3b-90b9-91c3d38c9302 service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:59.875358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-29abd503-948d-4d3b-90b9-91c3d38c9302 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:59.979149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abd503-948d-4d3b-90b9-91c3d38c9302 service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:59.984712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc6a53-a692-4989-9ca4-8f63dd42806a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:59.985418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc6a53-a692-4989-9ca4-8f63dd42806a service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:59.985918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc6a53-a692-4989-9ca4-8f63dd42806a service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:59.986051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc6a53-a692-4989-9ca4-8f63dd42806a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:59.990801 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-85bc6a53-a692-4989-9ca4-8f63dd42806a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:00.004961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc6a53-a692-4989-9ca4-8f63dd42806a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:00.006713 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc6a53-a692-4989-9ca4-8f63dd42806a service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:00.018165 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1309/2594] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:03.253505 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f02c33-8069-47cc-bb02-8b6f31b3305b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:03.265041 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f02c33-8069-47cc-bb02-8b6f31b3305b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:03.270145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b41dc-d47d-4844-8219-f810245bf3b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:03.271252 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b41dc-d47d-4844-8219-f810245bf3b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:03.271814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b41dc-d47d-4844-8219-f810245bf3b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:03.272153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b41dc-d47d-4844-8219-f810245bf3b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:03.273957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3f0b41dc-d47d-4844-8219-f810245bf3b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:03.284611 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b41dc-d47d-4844-8219-f810245bf3b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:03.286899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b41dc-d47d-4844-8219-f810245bf3b3 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:03.295486 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1286/2595] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:10.012957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=64 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:13.291386 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=71 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:17.713557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc6a53-a692-4989-9ca4-8f63dd42806a service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:17.715087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-85bc6a53-a692-4989-9ca4-8f63dd42806a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:17.776020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc6a53-a692-4989-9ca4-8f63dd42806a service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:17.779885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4f123-079a-4ad5-8deb-522495c8497a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af69888b04c45b09e34d9ff7a0b564f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:17.780538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4f123-079a-4ad5-8deb-522495c8497a service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:17.780824 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4f123-079a-4ad5-8deb-522495c8497a service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:17.781101 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4f123-079a-4ad5-8deb-522495c8497a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:17.782944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-59c4f123-079a-4ad5-8deb-522495c8497a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:17.796718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4f123-079a-4ad5-8deb-522495c8497a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:17.798154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4f123-079a-4ad5-8deb-522495c8497a service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:17.809512 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1310/2596] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:17.891193 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b41dc-d47d-4844-8219-f810245bf3b3 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:17.900410 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b41dc-d47d-4844-8219-f810245bf3b3 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:17.904136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7233aa7-c3dd-4c75-bae2-cab1ea66e72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:17.904926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7233aa7-c3dd-4c75-bae2-cab1ea66e72f service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:17.905435 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7233aa7-c3dd-4c75-bae2-cab1ea66e72f service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:17.905712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7233aa7-c3dd-4c75-bae2-cab1ea66e72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:17.907480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e7233aa7-c3dd-4c75-bae2-cab1ea66e72f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:17.917992 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7233aa7-c3dd-4c75-bae2-cab1ea66e72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:17.919431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7233aa7-c3dd-4c75-bae2-cab1ea66e72f service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:17.927298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1287/2597] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:12:18.000894 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4f123-079a-4ad5-8deb-522495c8497a service glance] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:18.002309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-59c4f123-079a-4ad5-8deb-522495c8497a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:18.014596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4f123-079a-4ad5-8deb-522495c8497a service glance] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:18.030830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386e979-c07b-40d1-a7b0-fa70414f6c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:18.030830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386e979-c07b-40d1-a7b0-fa70414f6c55 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:18.030830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386e979-c07b-40d1-a7b0-fa70414f6c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:18.030830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386e979-c07b-40d1-a7b0-fa70414f6c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:18.031693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c386e979-c07b-40d1-a7b0-fa70414f6c55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:18.036220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386e979-c07b-40d1-a7b0-fa70414f6c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:18.037640 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386e979-c07b-40d1-a7b0-fa70414f6c55 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:18.048424 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1311/2598] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:12:21.278942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e972e9-c419-4fb2-8043-65915372d5c3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:21.279559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e972e9-c419-4fb2-8043-65915372d5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:21.279676 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e972e9-c419-4fb2-8043-65915372d5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:21.299770 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a9e972e9-c419-4fb2-8043-65915372d5c3 None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:21.302285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a9e972e9-c419-4fb2-8043-65915372d5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:21.310379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1288/2599] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:12:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:21.324398 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386e979-c07b-40d1-a7b0-fa70414f6c55 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:21.349204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386e979-c07b-40d1-a7b0-fa70414f6c55 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:21.354299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43921586-6c75-4e41-8227-356bd36f741e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:21.354901 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43921586-6c75-4e41-8227-356bd36f741e service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:21.355211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43921586-6c75-4e41-8227-356bd36f741e service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:21.355486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43921586-6c75-4e41-8227-356bd36f741e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:21.366747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43921586-6c75-4e41-8227-356bd36f741e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:21.368358 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43921586-6c75-4e41-8227-356bd36f741e service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:21.377793 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1312/2600] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:12:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:12:31.294681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=9 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:31.373829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=42 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:40.646253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e972e9-c419-4fb2-8043-65915372d5c3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:40.657085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e972e9-c419-4fb2-8043-65915372d5c3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:40.660951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c81c55-3e2e-4fa2-9b57-b720bc55d0b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:40.661644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c81c55-3e2e-4fa2-9b57-b720bc55d0b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:40.662095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c81c55-3e2e-4fa2-9b57-b720bc55d0b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:40.662372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c81c55-3e2e-4fa2-9b57-b720bc55d0b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:40.671686 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c81c55-3e2e-4fa2-9b57-b720bc55d0b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:40.673239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c81c55-3e2e-4fa2-9b57-b720bc55d0b2 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:40.687641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1289/2601] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:40.739220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43921586-6c75-4e41-8227-356bd36f741e service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:40.752528 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43921586-6c75-4e41-8227-356bd36f741e service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:40.758890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a289b-9daf-49dd-b0be-80b595359c25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:40.759542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a289b-9daf-49dd-b0be-80b595359c25 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:40.760127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a289b-9daf-49dd-b0be-80b595359c25 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:40.760870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a289b-9daf-49dd-b0be-80b595359c25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:40.764007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-010a289b-9daf-49dd-b0be-80b595359c25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:40.780586 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a289b-9daf-49dd-b0be-80b595359c25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:40.782327 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a289b-9daf-49dd-b0be-80b595359c25 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:40.794889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1313/2602] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:43.142449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355ac5-d30d-47f4-b912-4a8844c9f626 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:43.144399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355ac5-d30d-47f4-b912-4a8844c9f626 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:43.144964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355ac5-d30d-47f4-b912-4a8844c9f626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:43.165145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-38355ac5-d30d-47f4-b912-4a8844c9f626 None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:43.166419 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-38355ac5-d30d-47f4-b912-4a8844c9f626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:43.173740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1290/2603] 10.222.0.86 () {58 vars in 967 bytes} [Mon Aug 29 19:12:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:43.188355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a289b-9daf-49dd-b0be-80b595359c25 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:43.200934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a289b-9daf-49dd-b0be-80b595359c25 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:43.207704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35088631-7191-41ed-a301-fd483b8c1a47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:43.208404 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35088631-7191-41ed-a301-fd483b8c1a47 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:43.208846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35088631-7191-41ed-a301-fd483b8c1a47 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:43.209292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35088631-7191-41ed-a301-fd483b8c1a47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:43.219052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35088631-7191-41ed-a301-fd483b8c1a47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:43.220766 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35088631-7191-41ed-a301-fd483b8c1a47 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:43.228992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1314/2604] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:12:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 19:12:53.159846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=9 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:53.226013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=12 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:07.232966 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38355ac5-d30d-47f4-b912-4a8844c9f626 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:07.242897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38355ac5-d30d-47f4-b912-4a8844c9f626 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:07.246476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e9cfa-dad2-4fdc-99a9-d698ed0087aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:07.247106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e9cfa-dad2-4fdc-99a9-d698ed0087aa service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:07.247391 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e9cfa-dad2-4fdc-99a9-d698ed0087aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:07.247701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e9cfa-dad2-4fdc-99a9-d698ed0087aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:07.249391 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-196e9cfa-dad2-4fdc-99a9-d698ed0087aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:07.309802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e9cfa-dad2-4fdc-99a9-d698ed0087aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:07.311246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e9cfa-dad2-4fdc-99a9-d698ed0087aa service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:07.320675 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1291/2605] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:10.594364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35088631-7191-41ed-a301-fd483b8c1a47 service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:10.596039 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-35088631-7191-41ed-a301-fd483b8c1a47 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:10.610976 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35088631-7191-41ed-a301-fd483b8c1a47 service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:10.615574 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ae63a-5ca2-47cd-bf94-bd60beb6c6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:10.616182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ae63a-5ca2-47cd-bf94-bd60beb6c6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:10.616462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ae63a-5ca2-47cd-bf94-bd60beb6c6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:10.616735 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ae63a-5ca2-47cd-bf94-bd60beb6c6c1 admin admin] PATH_INFO: `/v3/users/b114570010d94eea955b7b3f1fa97c0a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:10.638193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ae63a-5ca2-47cd-bf94-bd60beb6c6c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b114570010d94eea955b7b3f1fa97c0a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:10.640176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ae63a-5ca2-47cd-bf94-bd60beb6c6c1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:10.689586 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba7ae63a-5ca2-47cd-bf94-bd60beb6c6c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b114570010d94eea955b7b3f1fa97c0a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:10.690273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba7ae63a-5ca2-47cd-bf94-bd60beb6c6c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b114570010d94eea955b7b3f1fa97c0a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:10.703356 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba7ae63a-5ca2-47cd-bf94-bd60beb6c6c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b114570010d94eea955b7b3f1fa97c0a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:10.711457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba7ae63a-5ca2-47cd-bf94-bd60beb6c6c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b114570010d94eea955b7b3f1fa97c0a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:10.718759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba7ae63a-5ca2-47cd-bf94-bd60beb6c6c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b114570010d94eea955b7b3f1fa97c0a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:10.725644 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1315/2606] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:13:10 2022] DELETE /identity/v3/users/b114570010d94eea955b7b3f1fa97c0a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:13:10.903541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e9cfa-dad2-4fdc-99a9-d698ed0087aa service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:10.915277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e9cfa-dad2-4fdc-99a9-d698ed0087aa service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:10.973101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:10.973732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:10.974007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:10.974279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] PATH_INFO: `/v3/projects/12c146a9d54f4ed0a4e8a4a835d6367f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:10.984568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12c146a9d54f4ed0a4e8a4a835d6367f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:10.986463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:11.032071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12c146a9d54f4ed0a4e8a4a835d6367f', 'request_id': 'req-c0273477-7903-41ae-86dc-b84f0ab81d14'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:11.038493 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12c146a9d54f4ed0a4e8a4a835d6367f', 'request_id': 'req-c0273477-7903-41ae-86dc-b84f0ab81d14'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:11.044011 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12c146a9d54f4ed0a4e8a4a835d6367f', 'request_id': 'req-c0273477-7903-41ae-86dc-b84f0ab81d14'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:11.058169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] The token cache is being invalidate because project 12c146a9d54f4ed0a4e8a4a835d6367f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:13:11.058456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:11.059101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0273477-7903-41ae-86dc-b84f0ab81d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:11.060763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1292/2607] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:13:10 2022] DELETE /identity/v3/projects/12c146a9d54f4ed0a4e8a4a835d6367f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:13:11.066434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db65b35-289d-4efb-8474-80b0030cee40 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.066727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db65b35-289d-4efb-8474-80b0030cee40 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.067028 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db65b35-289d-4efb-8474-80b0030cee40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.089217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0db65b35-289d-4efb-8474-80b0030cee40 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:11.090465 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0db65b35-289d-4efb-8474-80b0030cee40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:11.150812 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1316/2608] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:11.159402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317b75a-d797-4463-9e54-50c42f039b77 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.159687 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317b75a-d797-4463-9e54-50c42f039b77 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.159986 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317b75a-d797-4463-9e54-50c42f039b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.180025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a317b75a-d797-4463-9e54-50c42f039b77 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:11.181231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a317b75a-d797-4463-9e54-50c42f039b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:11.189197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1293/2609] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:11.194128 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db65b35-289d-4efb-8474-80b0030cee40 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:11.204293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db65b35-289d-4efb-8474-80b0030cee40 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:11.209410 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640b215-5d4d-4025-9a40-61a177c46d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:11.210000 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640b215-5d4d-4025-9a40-61a177c46d55 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.210282 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640b215-5d4d-4025-9a40-61a177c46d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.210579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640b215-5d4d-4025-9a40-61a177c46d55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.211275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640b215-5d4d-4025-9a40-61a177c46d55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:13:11.211578 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640b215-5d4d-4025-9a40-61a177c46d55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:11.213176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640b215-5d4d-4025-9a40-61a177c46d55 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:11.224763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1317/2610] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:13:11.229210 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317b75a-d797-4463-9e54-50c42f039b77 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:11.239185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317b75a-d797-4463-9e54-50c42f039b77 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:11.243839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae516c74-2c53-4044-8b30-94195fc5c158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:11.244438 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae516c74-2c53-4044-8b30-94195fc5c158 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.244718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae516c74-2c53-4044-8b30-94195fc5c158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.244990 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae516c74-2c53-4044-8b30-94195fc5c158 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.245612 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae516c74-2c53-4044-8b30-94195fc5c158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:11.247558 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae516c74-2c53-4044-8b30-94195fc5c158 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:11.253284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:11.286167 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1294/2611] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:13:11 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:13:11.291254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640b215-5d4d-4025-9a40-61a177c46d55 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:11.302050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640b215-5d4d-4025-9a40-61a177c46d55 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:11.348107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:11.364308 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be5db0-f7c6-41b0-9be7-bcfb2370e60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:11.364919 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be5db0-f7c6-41b0-9be7-bcfb2370e60a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.365199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be5db0-f7c6-41b0-9be7-bcfb2370e60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.365475 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be5db0-f7c6-41b0-9be7-bcfb2370e60a admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.366104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be5db0-f7c6-41b0-9be7-bcfb2370e60a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:11.368189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be5db0-f7c6-41b0-9be7-bcfb2370e60a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:11.391385 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1318/2612] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:13:11 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:13:11.396718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae516c74-2c53-4044-8b30-94195fc5c158 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:11.407107 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae516c74-2c53-4044-8b30-94195fc5c158 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:11.411864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef447839-6c54-4cdc-aa1a-91b4b8d0191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:11.412468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef447839-6c54-4cdc-aa1a-91b4b8d0191a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.412746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef447839-6c54-4cdc-aa1a-91b4b8d0191a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.413015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef447839-6c54-4cdc-aa1a-91b4b8d0191a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.413656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef447839-6c54-4cdc-aa1a-91b4b8d0191a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:11.415266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef447839-6c54-4cdc-aa1a-91b4b8d0191a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:11.426012 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1295/2613] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:13:11.430267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be5db0-f7c6-41b0-9be7-bcfb2370e60a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:11.441137 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be5db0-f7c6-41b0-9be7-bcfb2370e60a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:11.445749 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191a4f8-1216-486c-9fb9-875d6c640548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:11.446343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a4f8-1216-486c-9fb9-875d6c640548 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.446620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a4f8-1216-486c-9fb9-875d6c640548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.446889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a4f8-1216-486c-9fb9-875d6c640548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.447547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191a4f8-1216-486c-9fb9-875d6c640548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:11.449121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191a4f8-1216-486c-9fb9-875d6c640548 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:11.461031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1319/2614] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:13:11.465269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef447839-6c54-4cdc-aa1a-91b4b8d0191a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:11.475951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef447839-6c54-4cdc-aa1a-91b4b8d0191a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:11.480523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f6440-3452-4f73-b5db-02bc3d38d94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:11.481260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f6440-3452-4f73-b5db-02bc3d38d94c admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.481538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f6440-3452-4f73-b5db-02bc3d38d94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.481809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f6440-3452-4f73-b5db-02bc3d38d94c admin admin] PATH_INFO: `/v3/projects/fe9fc68075be4712a305fb8a15c91f19/users/a982dbba4a324a4f893e427000ce9b93/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.497010 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f6440-3452-4f73-b5db-02bc3d38d94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9fc68075be4712a305fb8a15c91f19, user_id=a982dbba4a324a4f893e427000ce9b93, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:11.500456 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f6440-3452-4f73-b5db-02bc3d38d94c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:11.514445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1296/2615] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:13:11 2022] PUT /identity/v3/projects/fe9fc68075be4712a305fb8a15c91f19/users/a982dbba4a324a4f893e427000ce9b93/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:13:11.520155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ca48b-c6af-465b-b874-e9d86d044d5c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.520423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ca48b-c6af-465b-b874-e9d86d044d5c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.520693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ca48b-c6af-465b-b874-e9d86d044d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.540788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-955ca48b-c6af-465b-b874-e9d86d044d5c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:11.541899 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-955ca48b-c6af-465b-b874-e9d86d044d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:11.600868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1320/2616] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:11.610723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f6440-3452-4f73-b5db-02bc3d38d94c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:11.612475 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-326f6440-3452-4f73-b5db-02bc3d38d94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:11.670680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f6440-3452-4f73-b5db-02bc3d38d94c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:11.674393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55b76f-35c3-4537-83eb-2c5c4134c7fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:11.675007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55b76f-35c3-4537-83eb-2c5c4134c7fc service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:11.675314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55b76f-35c3-4537-83eb-2c5c4134c7fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:11.675587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55b76f-35c3-4537-83eb-2c5c4134c7fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:11.684980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55b76f-35c3-4537-83eb-2c5c4134c7fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:11.686395 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55b76f-35c3-4537-83eb-2c5c4134c7fc service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:11.696720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1297/2617] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:13:16.049482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e2b11-fdda-4a6b-a94b-8a9f6ec2a8da None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:16.049991 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e2b11-fdda-4a6b-a94b-8a9f6ec2a8da None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:16.050099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e2b11-fdda-4a6b-a94b-8a9f6ec2a8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:16.081823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1a0e2b11-fdda-4a6b-a94b-8a9f6ec2a8da None None] MFA Rules not processed for user `a982dbba4a324a4f893e427000ce9b93`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:16.083108 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1a0e2b11-fdda-4a6b-a94b-8a9f6ec2a8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:16.161217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1321/2618] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:13:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:16.176135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55b76f-35c3-4537-83eb-2c5c4134c7fc service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:16.178019 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5c55b76f-35c3-4537-83eb-2c5c4134c7fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:16.265701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55b76f-35c3-4537-83eb-2c5c4134c7fc service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:16.272762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7486a-b9f8-4439-adc9-ae957dc10c2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:16.273399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7486a-b9f8-4439-adc9-ae957dc10c2a service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:16.273677 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7486a-b9f8-4439-adc9-ae957dc10c2a service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:16.273949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7486a-b9f8-4439-adc9-ae957dc10c2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:16.282859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7486a-b9f8-4439-adc9-ae957dc10c2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:16.285054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7486a-b9f8-4439-adc9-ae957dc10c2a service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:16.308193 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1298/2619] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:17.552222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8075e59-b4aa-491d-9ffd-36edc15f2908 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:17.552723 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8075e59-b4aa-491d-9ffd-36edc15f2908 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:17.552831 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8075e59-b4aa-491d-9ffd-36edc15f2908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:17.572846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c8075e59-b4aa-491d-9ffd-36edc15f2908 None None] MFA Rules not processed for user `a982dbba4a324a4f893e427000ce9b93`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:17.574057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c8075e59-b4aa-491d-9ffd-36edc15f2908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:17.581472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1322/2620] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:13:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:26.291292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:27.567247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:12.984221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7486a-b9f8-4439-adc9-ae957dc10c2a service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:12.994409 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7486a-b9f8-4439-adc9-ae957dc10c2a service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:12.999110 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566c5a6-bdab-43a8-b55e-5ed5c14d72a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:12.999720 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566c5a6-bdab-43a8-b55e-5ed5c14d72a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.000040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566c5a6-bdab-43a8-b55e-5ed5c14d72a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.000315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566c5a6-bdab-43a8-b55e-5ed5c14d72a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.002800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c566c5a6-bdab-43a8-b55e-5ed5c14d72a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:13.016931 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566c5a6-bdab-43a8-b55e-5ed5c14d72a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:13.018458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566c5a6-bdab-43a8-b55e-5ed5c14d72a3 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:13.028533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1299/2621] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:16.003939 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8075e59-b4aa-491d-9ffd-36edc15f2908 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:16.005524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c8075e59-b4aa-491d-9ffd-36edc15f2908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:16.068014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566c5a6-bdab-43a8-b55e-5ed5c14d72a3 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:16.069887 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c566c5a6-bdab-43a8-b55e-5ed5c14d72a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:16.086763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8075e59-b4aa-491d-9ffd-36edc15f2908 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:16.088685 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566c5a6-bdab-43a8-b55e-5ed5c14d72a3 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:16.092493 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49370bce-3053-435a-90d4-19d601d2eddd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:16.093151 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49370bce-3053-435a-90d4-19d601d2eddd service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.093449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49370bce-3053-435a-90d4-19d601d2eddd service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.093731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49370bce-3053-435a-90d4-19d601d2eddd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.094791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29408164-0d0e-48c3-b74b-cf5d3853225e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:16.095477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29408164-0d0e-48c3-b74b-cf5d3853225e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.095760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29408164-0d0e-48c3-b74b-cf5d3853225e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.096039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29408164-0d0e-48c3-b74b-cf5d3853225e admin admin] PATH_INFO: `/v3/users/dd013ba83e944ecfa1a2d0a8a0d55101` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.096179 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-49370bce-3053-435a-90d4-19d601d2eddd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:16.097173 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29408164-0d0e-48c3-b74b-cf5d3853225e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd013ba83e944ecfa1a2d0a8a0d55101)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:16.103560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29408164-0d0e-48c3-b74b-cf5d3853225e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:16.108449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49370bce-3053-435a-90d4-19d601d2eddd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:16.110584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49370bce-3053-435a-90d4-19d601d2eddd service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:16.120667 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1323/2622] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:16.148354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29408164-0d0e-48c3-b74b-cf5d3853225e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd013ba83e944ecfa1a2d0a8a0d55101'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:16.162809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29408164-0d0e-48c3-b74b-cf5d3853225e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd013ba83e944ecfa1a2d0a8a0d55101'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:16.163555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29408164-0d0e-48c3-b74b-cf5d3853225e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd013ba83e944ecfa1a2d0a8a0d55101'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:16.164165 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29408164-0d0e-48c3-b74b-cf5d3853225e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd013ba83e944ecfa1a2d0a8a0d55101'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:16.173624 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29408164-0d0e-48c3-b74b-cf5d3853225e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd013ba83e944ecfa1a2d0a8a0d55101'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:16.187219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1300/2623] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:16 2022] DELETE /identity/v3/users/dd013ba83e944ecfa1a2d0a8a0d55101 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:16.375209 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49370bce-3053-435a-90d4-19d601d2eddd service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:16.390019 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49370bce-3053-435a-90d4-19d601d2eddd service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:16.448816 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:16.449472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.449754 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.450113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] PATH_INFO: `/v3/projects/b3171bab0ea547d9acc3559f449221eb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.451571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3171bab0ea547d9acc3559f449221eb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:16.453533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:16.502935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3171bab0ea547d9acc3559f449221eb', 'request_id': 'req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:16.510220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3171bab0ea547d9acc3559f449221eb', 'request_id': 'req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:16.517127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3171bab0ea547d9acc3559f449221eb', 'request_id': 'req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:16.527642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] The token cache is being invalidate because project b3171bab0ea547d9acc3559f449221eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:16.527964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:16.528674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ba926ee5-4a9f-4f44-bf4a-5f2e28aa002c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:16.530302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1324/2624] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:16 2022] DELETE /identity/v3/projects/b3171bab0ea547d9acc3559f449221eb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:16.535236 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cde523-99c1-492e-aba1-4d395750b109 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.535528 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cde523-99c1-492e-aba1-4d395750b109 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.535827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cde523-99c1-492e-aba1-4d395750b109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.556307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e2cde523-99c1-492e-aba1-4d395750b109 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:16.557627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e2cde523-99c1-492e-aba1-4d395750b109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:16.617018 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1301/2625] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:16.624921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec645f-358a-4f01-915d-2cd2fd369c52 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.625250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec645f-358a-4f01-915d-2cd2fd369c52 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.625533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec645f-358a-4f01-915d-2cd2fd369c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.646159 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d4ec645f-358a-4f01-915d-2cd2fd369c52 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:16.647400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d4ec645f-358a-4f01-915d-2cd2fd369c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:16.655264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1325/2626] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:16.659522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cde523-99c1-492e-aba1-4d395750b109 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:16.669107 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cde523-99c1-492e-aba1-4d395750b109 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:16.674252 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55acb52-c2c9-4e82-bb1f-63d474966e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:16.674842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55acb52-c2c9-4e82-bb1f-63d474966e13 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.675150 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55acb52-c2c9-4e82-bb1f-63d474966e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.675422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55acb52-c2c9-4e82-bb1f-63d474966e13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.676062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55acb52-c2c9-4e82-bb1f-63d474966e13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:16.676364 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55acb52-c2c9-4e82-bb1f-63d474966e13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:16.677951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55acb52-c2c9-4e82-bb1f-63d474966e13 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:16.690796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1302/2627] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:16.695124 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec645f-358a-4f01-915d-2cd2fd369c52 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:16.704652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec645f-358a-4f01-915d-2cd2fd369c52 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:16.709344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5b2c6-1938-4ec4-99f5-2d0dd575ceef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:16.709938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5b2c6-1938-4ec4-99f5-2d0dd575ceef admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.710216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5b2c6-1938-4ec4-99f5-2d0dd575ceef admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.710488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5b2c6-1938-4ec4-99f5-2d0dd575ceef admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.711165 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b5b2c6-1938-4ec4-99f5-2d0dd575ceef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:16.713056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b5b2c6-1938-4ec4-99f5-2d0dd575ceef admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:16.724215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:16.741631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1326/2628] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:16 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:16.745877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55acb52-c2c9-4e82-bb1f-63d474966e13 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:16.757221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55acb52-c2c9-4e82-bb1f-63d474966e13 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:16.801005 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:16.815591 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3d919-45c4-41de-8965-cd80b6b993c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:16.816228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3d919-45c4-41de-8965-cd80b6b993c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.816545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3d919-45c4-41de-8965-cd80b6b993c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.816823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3d919-45c4-41de-8965-cd80b6b993c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.817696 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3d919-45c4-41de-8965-cd80b6b993c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:16.819843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3d919-45c4-41de-8965-cd80b6b993c5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:16.841380 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1303/2629] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:16 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:16.846221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5b2c6-1938-4ec4-99f5-2d0dd575ceef admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:16.856620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5b2c6-1938-4ec4-99f5-2d0dd575ceef admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:16.861305 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40b2b6-0cab-4f91-b60b-8473828d739f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:16.862012 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40b2b6-0cab-4f91-b60b-8473828d739f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.862321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40b2b6-0cab-4f91-b60b-8473828d739f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.862599 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40b2b6-0cab-4f91-b60b-8473828d739f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.863323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40b2b6-0cab-4f91-b60b-8473828d739f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:16.864982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40b2b6-0cab-4f91-b60b-8473828d739f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:16.875463 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1327/2630] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:16.879809 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3d919-45c4-41de-8965-cd80b6b993c5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:16.889764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3d919-45c4-41de-8965-cd80b6b993c5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:16.896078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00d9ab-596b-4fab-9f48-f23ae3671623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:16.896627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00d9ab-596b-4fab-9f48-f23ae3671623 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.896911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00d9ab-596b-4fab-9f48-f23ae3671623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.897205 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00d9ab-596b-4fab-9f48-f23ae3671623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.897836 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00d9ab-596b-4fab-9f48-f23ae3671623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:16.899471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00d9ab-596b-4fab-9f48-f23ae3671623 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:16.909975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1304/2631] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:16.914276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40b2b6-0cab-4f91-b60b-8473828d739f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:16.924447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40b2b6-0cab-4f91-b60b-8473828d739f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:16.929988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c5be-a714-4078-b5c7-ad1a1c1ac51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:16.930631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c5be-a714-4078-b5c7-ad1a1c1ac51d admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.930907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c5be-a714-4078-b5c7-ad1a1c1ac51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.931214 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c5be-a714-4078-b5c7-ad1a1c1ac51d admin admin] PATH_INFO: `/v3/projects/dd195722fcbe47219220b5d222d338a1/users/755a22ec0e0d48349f6fcdaccf87de32/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.946638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208c5be-a714-4078-b5c7-ad1a1c1ac51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd195722fcbe47219220b5d222d338a1, user_id=755a22ec0e0d48349f6fcdaccf87de32, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:16.949451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208c5be-a714-4078-b5c7-ad1a1c1ac51d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:16.963240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1328/2632] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:16 2022] PUT /identity/v3/projects/dd195722fcbe47219220b5d222d338a1/users/755a22ec0e0d48349f6fcdaccf87de32/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:16.975198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21acbe-e68e-4093-8d91-d80db71b1f7f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.975491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21acbe-e68e-4093-8d91-d80db71b1f7f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.975768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21acbe-e68e-4093-8d91-d80db71b1f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.003476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5e21acbe-e68e-4093-8d91-d80db71b1f7f None None] MFA Rules not processed for user `755a22ec0e0d48349f6fcdaccf87de32`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:17.004722 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e21acbe-e68e-4093-8d91-d80db71b1f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:17.067311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1305/2633] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:17.071769 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c5be-a714-4078-b5c7-ad1a1c1ac51d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.132517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c5be-a714-4078-b5c7-ad1a1c1ac51d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:17.137038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62f121-3aa9-44dd-a65c-04321edc9786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:17.137633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62f121-3aa9-44dd-a65c-04321edc9786 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.137911 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62f121-3aa9-44dd-a65c-04321edc9786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.138182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62f121-3aa9-44dd-a65c-04321edc9786 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.138807 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62f121-3aa9-44dd-a65c-04321edc9786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:17.140734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62f121-3aa9-44dd-a65c-04321edc9786 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:17.167800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1329/2634] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:17 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:17.171883 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21acbe-e68e-4093-8d91-d80db71b1f7f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.182941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21acbe-e68e-4093-8d91-d80db71b1f7f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:17.240624 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f6e63-a967-4913-8f5c-3a4297bf8dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:17.241219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f6e63-a967-4913-8f5c-3a4297bf8dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.241495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f6e63-a967-4913-8f5c-3a4297bf8dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.241764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f6e63-a967-4913-8f5c-3a4297bf8dd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.242375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f6e63-a967-4913-8f5c-3a4297bf8dd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:17.244307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f6e63-a967-4913-8f5c-3a4297bf8dd1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:17.266156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1306/2635] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:17 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:17.270085 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62f121-3aa9-44dd-a65c-04321edc9786 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.280751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62f121-3aa9-44dd-a65c-04321edc9786 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:17.285457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1c164-ec2d-4011-b9c7-86f83407ec2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:17.286044 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1c164-ec2d-4011-b9c7-86f83407ec2d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.286322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1c164-ec2d-4011-b9c7-86f83407ec2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.286591 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1c164-ec2d-4011-b9c7-86f83407ec2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.287245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1c164-ec2d-4011-b9c7-86f83407ec2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:17.288827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1c164-ec2d-4011-b9c7-86f83407ec2d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:17.299954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1330/2636] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:17.303839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f6e63-a967-4913-8f5c-3a4297bf8dd1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.314502 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f6e63-a967-4913-8f5c-3a4297bf8dd1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:17.319242 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87e28e-1523-4202-9010-9d3b9f4950dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:17.319886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87e28e-1523-4202-9010-9d3b9f4950dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.320169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87e28e-1523-4202-9010-9d3b9f4950dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.320444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87e28e-1523-4202-9010-9d3b9f4950dd admin admin] PATH_INFO: `/v3/projects/8c6c99c31c554e48b7646dc2cc16856d/users/048fcc6c5add4f338a378cd7be789bae/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.335480 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87e28e-1523-4202-9010-9d3b9f4950dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c6c99c31c554e48b7646dc2cc16856d, user_id=048fcc6c5add4f338a378cd7be789bae, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:17.338191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87e28e-1523-4202-9010-9d3b9f4950dd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:17.351124 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1307/2637] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:17 2022] PUT /identity/v3/projects/8c6c99c31c554e48b7646dc2cc16856d/users/048fcc6c5add4f338a378cd7be789bae/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:17.354972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1c164-ec2d-4011-b9c7-86f83407ec2d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.365348 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1c164-ec2d-4011-b9c7-86f83407ec2d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:17.420445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e3278-acb5-4b2c-ba68-4deb9c5a158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:17.420973 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e3278-acb5-4b2c-ba68-4deb9c5a158b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.421257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e3278-acb5-4b2c-ba68-4deb9c5a158b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.421527 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e3278-acb5-4b2c-ba68-4deb9c5a158b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.422155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e3278-acb5-4b2c-ba68-4deb9c5a158b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:17.424310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e3278-acb5-4b2c-ba68-4deb9c5a158b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:17.433776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1331/2638] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:17.437996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87e28e-1523-4202-9010-9d3b9f4950dd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.448701 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87e28e-1523-4202-9010-9d3b9f4950dd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:17.453592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd3a41-9ebb-40a0-94df-83292141b471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:17.454332 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd3a41-9ebb-40a0-94df-83292141b471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.454612 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd3a41-9ebb-40a0-94df-83292141b471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.454886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd3a41-9ebb-40a0-94df-83292141b471 admin admin] PATH_INFO: `/v3/projects/8c6c99c31c554e48b7646dc2cc16856d/users/048fcc6c5add4f338a378cd7be789bae/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.456819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd3a41-9ebb-40a0-94df-83292141b471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c6c99c31c554e48b7646dc2cc16856d, user_id=048fcc6c5add4f338a378cd7be789bae, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:17.459507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd3a41-9ebb-40a0-94df-83292141b471 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:17.469723 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1308/2639] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:17 2022] PUT /identity/v3/projects/8c6c99c31c554e48b7646dc2cc16856d/users/048fcc6c5add4f338a378cd7be789bae/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:17.479727 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65461e01-a71e-4c39-9196-c69fd3c1ef34 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.480017 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65461e01-a71e-4c39-9196-c69fd3c1ef34 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.480292 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65461e01-a71e-4c39-9196-c69fd3c1ef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.508914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-65461e01-a71e-4c39-9196-c69fd3c1ef34 None None] MFA Rules not processed for user `048fcc6c5add4f338a378cd7be789bae`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:17.510119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-65461e01-a71e-4c39-9196-c69fd3c1ef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:17.522439 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:17.578144 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1332/2640] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:17.589199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd3a41-9ebb-40a0-94df-83292141b471 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.591009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f7bd3a41-9ebb-40a0-94df-83292141b471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:17.601222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:17.649281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd3a41-9ebb-40a0-94df-83292141b471 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:17.652963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b932f0e-4c68-446b-8348-07a9eec7473d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:17.653589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b932f0e-4c68-446b-8348-07a9eec7473d service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.653865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b932f0e-4c68-446b-8348-07a9eec7473d service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.654135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b932f0e-4c68-446b-8348-07a9eec7473d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.663520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b932f0e-4c68-446b-8348-07a9eec7473d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:17.664935 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b932f0e-4c68-446b-8348-07a9eec7473d service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:17.675029 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1309/2641] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:17.749194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65461e01-a71e-4c39-9196-c69fd3c1ef34 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.758793 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65461e01-a71e-4c39-9196-c69fd3c1ef34 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:17.762394 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d914f-5bae-4898-8ae7-7a9803f841d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:17.763029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d914f-5bae-4898-8ae7-7a9803f841d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.763311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d914f-5bae-4898-8ae7-7a9803f841d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.763584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d914f-5bae-4898-8ae7-7a9803f841d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.821401 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d914f-5bae-4898-8ae7-7a9803f841d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:17.822840 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d914f-5bae-4898-8ae7-7a9803f841d8 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:17.831895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1333/2642] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:20.540798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b932f0e-4c68-446b-8348-07a9eec7473d service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:20.604939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b932f0e-4c68-446b-8348-07a9eec7473d service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:20.609922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:20.611077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.611361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.611634 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] PATH_INFO: `/v3/users/755a22ec0e0d48349f6fcdaccf87de32` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.612748 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=755a22ec0e0d48349f6fcdaccf87de32)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:20.614695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:20.661236 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755a22ec0e0d48349f6fcdaccf87de32'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:20.673618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755a22ec0e0d48349f6fcdaccf87de32'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:20.674354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755a22ec0e0d48349f6fcdaccf87de32'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:20.674894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755a22ec0e0d48349f6fcdaccf87de32'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:20.684288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755a22ec0e0d48349f6fcdaccf87de32'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:20.698405 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1310/2643] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:20 2022] DELETE /identity/v3/users/755a22ec0e0d48349f6fcdaccf87de32 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:20.703347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7f5da-ea9b-4246-9c3d-413163f2c979 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.703867 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7f5da-ea9b-4246-9c3d-413163f2c979 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.703986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7f5da-ea9b-4246-9c3d-413163f2c979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.724273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-16e7f5da-ea9b-4246-9c3d-413163f2c979 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:20.725500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-16e7f5da-ea9b-4246-9c3d-413163f2c979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:20.783300 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1334/2644] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:20.793163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:20.794609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:20.828168 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:20.855748 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794adaa8-b7b0-49bd-b534-5f8be544a1ff admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:20.859493 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8489ee-5ac9-4280-a835-98f945516961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:20.860637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8489ee-5ac9-4280-a835-98f945516961 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.860913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8489ee-5ac9-4280-a835-98f945516961 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.861184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8489ee-5ac9-4280-a835-98f945516961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.869915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8489ee-5ac9-4280-a835-98f945516961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:20.871360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8489ee-5ac9-4280-a835-98f945516961 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:20.880693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1311/2645] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:21.155013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7f5da-ea9b-4246-9c3d-413163f2c979 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:21.156758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-16e7f5da-ea9b-4246-9c3d-413163f2c979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:21.171181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7f5da-ea9b-4246-9c3d-413163f2c979 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:21.175717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:21.176903 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:21.177371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:21.177648 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] PATH_INFO: `/v3/projects/dd195722fcbe47219220b5d222d338a1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:21.178833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd195722fcbe47219220b5d222d338a1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:21.180689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:21.225979 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:21.229230 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd195722fcbe47219220b5d222d338a1', 'request_id': 'req-b924372b-92ab-4b8e-a162-41ae45acdea8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.236506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd195722fcbe47219220b5d222d338a1', 'request_id': 'req-b924372b-92ab-4b8e-a162-41ae45acdea8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.242719 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd195722fcbe47219220b5d222d338a1', 'request_id': 'req-b924372b-92ab-4b8e-a162-41ae45acdea8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.253224 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] The token cache is being invalidate because project dd195722fcbe47219220b5d222d338a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:21.253504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.254119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.255707 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1335/2646] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:21 2022] DELETE /identity/v3/projects/dd195722fcbe47219220b5d222d338a1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:21.259741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8489ee-5ac9-4280-a835-98f945516961 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:21.270032 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8489ee-5ac9-4280-a835-98f945516961 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:21.325008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e281dca-5e7e-49e1-984c-f31c32f6b402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:21.325627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e281dca-5e7e-49e1-984c-f31c32f6b402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:21.326161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e281dca-5e7e-49e1-984c-f31c32f6b402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:21.326433 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e281dca-5e7e-49e1-984c-f31c32f6b402 admin admin] PATH_INFO: `/v3/users/048fcc6c5add4f338a378cd7be789bae` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:21.327566 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e281dca-5e7e-49e1-984c-f31c32f6b402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=048fcc6c5add4f338a378cd7be789bae)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:21.329952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e281dca-5e7e-49e1-984c-f31c32f6b402 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:21.373558 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e281dca-5e7e-49e1-984c-f31c32f6b402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '048fcc6c5add4f338a378cd7be789bae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.385568 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e281dca-5e7e-49e1-984c-f31c32f6b402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '048fcc6c5add4f338a378cd7be789bae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.386212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e281dca-5e7e-49e1-984c-f31c32f6b402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '048fcc6c5add4f338a378cd7be789bae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.386830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e281dca-5e7e-49e1-984c-f31c32f6b402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '048fcc6c5add4f338a378cd7be789bae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.394641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e281dca-5e7e-49e1-984c-f31c32f6b402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '048fcc6c5add4f338a378cd7be789bae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.406551 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1312/2647] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:21 2022] DELETE /identity/v3/users/048fcc6c5add4f338a378cd7be789bae => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:21.673861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:21.675478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:21.700557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924372b-92ab-4b8e-a162-41ae45acdea8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:21.757708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:21.758325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:21.758598 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:21.758866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] PATH_INFO: `/v3/projects/8c6c99c31c554e48b7646dc2cc16856d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:21.760029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c6c99c31c554e48b7646dc2cc16856d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:21.761909 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:21.806486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c6c99c31c554e48b7646dc2cc16856d', 'request_id': 'req-dd971a1f-2466-4014-8148-1769eecf60b8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.815777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c6c99c31c554e48b7646dc2cc16856d', 'request_id': 'req-dd971a1f-2466-4014-8148-1769eecf60b8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.822120 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c6c99c31c554e48b7646dc2cc16856d', 'request_id': 'req-dd971a1f-2466-4014-8148-1769eecf60b8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.832393 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] The token cache is being invalidate because project 8c6c99c31c554e48b7646dc2cc16856d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:21.832677 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.833301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd971a1f-2466-4014-8148-1769eecf60b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:21.834750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1336/2648] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:21 2022] DELETE /identity/v3/projects/8c6c99c31c554e48b7646dc2cc16856d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:21.839681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5888edd-124a-4029-8fac-a5c23d706845 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:21.840157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5888edd-124a-4029-8fac-a5c23d706845 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:21.840271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5888edd-124a-4029-8fac-a5c23d706845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:21.859651 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d5888edd-124a-4029-8fac-a5c23d706845 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:21.861341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d5888edd-124a-4029-8fac-a5c23d706845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:21.904930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:21.920046 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1313/2649] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:21.928710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2568a2-52be-411b-aa6a-b1470a797cc8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:21.928995 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2568a2-52be-411b-aa6a-b1470a797cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:21.929295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2568a2-52be-411b-aa6a-b1470a797cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:21.949571 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bd2568a2-52be-411b-aa6a-b1470a797cc8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:21.951254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bd2568a2-52be-411b-aa6a-b1470a797cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:21.959457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1337/2650] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:21.963690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5888edd-124a-4029-8fac-a5c23d706845 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:21.973718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5888edd-124a-4029-8fac-a5c23d706845 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:21.978257 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18a2ec-d2d9-445b-935b-de04c702e8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:21.978849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18a2ec-d2d9-445b-935b-de04c702e8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:21.979230 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18a2ec-d2d9-445b-935b-de04c702e8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:21.979506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18a2ec-d2d9-445b-935b-de04c702e8f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:21.980147 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18a2ec-d2d9-445b-935b-de04c702e8f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:21.980448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18a2ec-d2d9-445b-935b-de04c702e8f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:21.982022 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18a2ec-d2d9-445b-935b-de04c702e8f8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.002444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1314/2651] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:22.007105 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2568a2-52be-411b-aa6a-b1470a797cc8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.016418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2568a2-52be-411b-aa6a-b1470a797cc8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.021142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe2621-4473-4fc5-a589-eddaa84620f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.021731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe2621-4473-4fc5-a589-eddaa84620f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.022006 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe2621-4473-4fc5-a589-eddaa84620f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.022273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe2621-4473-4fc5-a589-eddaa84620f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.022905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe2621-4473-4fc5-a589-eddaa84620f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.024813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe2621-4473-4fc5-a589-eddaa84620f5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.051562 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1338/2652] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:22 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:22.055670 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18a2ec-d2d9-445b-935b-de04c702e8f8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.066267 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18a2ec-d2d9-445b-935b-de04c702e8f8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.123084 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a266058-30f8-4c0d-95ad-8b3215665f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.123693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a266058-30f8-4c0d-95ad-8b3215665f5c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.123977 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a266058-30f8-4c0d-95ad-8b3215665f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.124255 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a266058-30f8-4c0d-95ad-8b3215665f5c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.124892 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a266058-30f8-4c0d-95ad-8b3215665f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.126803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a266058-30f8-4c0d-95ad-8b3215665f5c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.148898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1315/2653] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:22 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:22.153118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe2621-4473-4fc5-a589-eddaa84620f5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.163796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe2621-4473-4fc5-a589-eddaa84620f5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.168368 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a69c6-bfd2-48fb-9d1a-f359d1e02205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.169150 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a69c6-bfd2-48fb-9d1a-f359d1e02205 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.169427 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a69c6-bfd2-48fb-9d1a-f359d1e02205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.169696 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a69c6-bfd2-48fb-9d1a-f359d1e02205 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.170322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a69c6-bfd2-48fb-9d1a-f359d1e02205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.171915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a69c6-bfd2-48fb-9d1a-f359d1e02205 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.182266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1339/2654] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:22.186692 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a266058-30f8-4c0d-95ad-8b3215665f5c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.196746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a266058-30f8-4c0d-95ad-8b3215665f5c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.201673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70998ab-c73a-4193-a182-bd24ee235318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.202261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70998ab-c73a-4193-a182-bd24ee235318 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.202536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70998ab-c73a-4193-a182-bd24ee235318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.202806 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70998ab-c73a-4193-a182-bd24ee235318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.203509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70998ab-c73a-4193-a182-bd24ee235318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.205053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70998ab-c73a-4193-a182-bd24ee235318 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.215453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1316/2655] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:22.220044 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a69c6-bfd2-48fb-9d1a-f359d1e02205 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.230126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a69c6-bfd2-48fb-9d1a-f359d1e02205 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.234771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b104b-3358-40dd-93e6-739d05738bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.236024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b104b-3358-40dd-93e6-739d05738bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.236306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b104b-3358-40dd-93e6-739d05738bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.236599 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b104b-3358-40dd-93e6-739d05738bcb admin admin] PATH_INFO: `/v3/projects/7bfabad318fd45589a834c5593fe7508/users/82232ea9c33f4bf19ad725cfed7c7f4a/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.250851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b104b-3358-40dd-93e6-739d05738bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bfabad318fd45589a834c5593fe7508, user_id=82232ea9c33f4bf19ad725cfed7c7f4a, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.253552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b104b-3358-40dd-93e6-739d05738bcb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.263511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:22.268868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1340/2656] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:22 2022] PUT /identity/v3/projects/7bfabad318fd45589a834c5593fe7508/users/82232ea9c33f4bf19ad725cfed7c7f4a/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:22.280212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0793a3-3c06-46c7-8a41-d26312742a40 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.280497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0793a3-3c06-46c7-8a41-d26312742a40 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.280772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0793a3-3c06-46c7-8a41-d26312742a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.308690 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ce0793a3-3c06-46c7-8a41-d26312742a40 None None] MFA Rules not processed for user `82232ea9c33f4bf19ad725cfed7c7f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:22.309921 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ce0793a3-3c06-46c7-8a41-d26312742a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:22.371478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1317/2657] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:22.375926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b104b-3358-40dd-93e6-739d05738bcb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.438189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b104b-3358-40dd-93e6-739d05738bcb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.442826 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ce2fd-466d-4be3-b58e-0e36d6ec2f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.443455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ce2fd-466d-4be3-b58e-0e36d6ec2f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.443736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ce2fd-466d-4be3-b58e-0e36d6ec2f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.444008 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ce2fd-466d-4be3-b58e-0e36d6ec2f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.444668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ce2fd-466d-4be3-b58e-0e36d6ec2f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.446546 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ce2fd-466d-4be3-b58e-0e36d6ec2f87 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.473460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1341/2658] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:22 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:22.477545 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0793a3-3c06-46c7-8a41-d26312742a40 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.488227 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0793a3-3c06-46c7-8a41-d26312742a40 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.531219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:22.544898 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116fd21-5ad5-4be8-a85c-e046a68ce0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.545507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116fd21-5ad5-4be8-a85c-e046a68ce0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.545785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116fd21-5ad5-4be8-a85c-e046a68ce0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.546058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116fd21-5ad5-4be8-a85c-e046a68ce0c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.546682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116fd21-5ad5-4be8-a85c-e046a68ce0c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.548682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116fd21-5ad5-4be8-a85c-e046a68ce0c0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.569028 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1318/2659] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:22 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:22.572937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ce2fd-466d-4be3-b58e-0e36d6ec2f87 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.583880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ce2fd-466d-4be3-b58e-0e36d6ec2f87 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.588467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b5af9-5ff7-40d1-bcde-3aa1c400c9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.589083 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b5af9-5ff7-40d1-bcde-3aa1c400c9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.589363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b5af9-5ff7-40d1-bcde-3aa1c400c9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.589637 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b5af9-5ff7-40d1-bcde-3aa1c400c9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.590269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b5af9-5ff7-40d1-bcde-3aa1c400c9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.591891 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b5af9-5ff7-40d1-bcde-3aa1c400c9d3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.603052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1342/2660] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:22.607111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116fd21-5ad5-4be8-a85c-e046a68ce0c0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.617107 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116fd21-5ad5-4be8-a85c-e046a68ce0c0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.622211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378be69c-c1d8-44de-a9bf-24db0807ae07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.622865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378be69c-c1d8-44de-a9bf-24db0807ae07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.623248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378be69c-c1d8-44de-a9bf-24db0807ae07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.623525 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378be69c-c1d8-44de-a9bf-24db0807ae07 admin admin] PATH_INFO: `/v3/projects/718d72079d934516a5927c2f8f5bf064/users/c0f6c8be888d4c7f81ca6b2244c5614c/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.640660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378be69c-c1d8-44de-a9bf-24db0807ae07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=718d72079d934516a5927c2f8f5bf064, user_id=c0f6c8be888d4c7f81ca6b2244c5614c, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.643856 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378be69c-c1d8-44de-a9bf-24db0807ae07 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.656415 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1319/2661] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:22 2022] PUT /identity/v3/projects/718d72079d934516a5927c2f8f5bf064/users/c0f6c8be888d4c7f81ca6b2244c5614c/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:22.661213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b5af9-5ff7-40d1-bcde-3aa1c400c9d3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.671120 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b5af9-5ff7-40d1-bcde-3aa1c400c9d3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.734449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c6db4-ab2c-4912-ba2f-e7a17238a622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.735607 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c6db4-ab2c-4912-ba2f-e7a17238a622 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.735886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c6db4-ab2c-4912-ba2f-e7a17238a622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.736154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c6db4-ab2c-4912-ba2f-e7a17238a622 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.736811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c6db4-ab2c-4912-ba2f-e7a17238a622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.738388 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c6db4-ab2c-4912-ba2f-e7a17238a622 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.748092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1343/2662] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:22.752912 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378be69c-c1d8-44de-a9bf-24db0807ae07 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.763400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378be69c-c1d8-44de-a9bf-24db0807ae07 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.768308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4380237-891a-4340-b43f-ddc5cbaa073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.768960 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4380237-891a-4340-b43f-ddc5cbaa073b admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.769236 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4380237-891a-4340-b43f-ddc5cbaa073b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.769507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4380237-891a-4340-b43f-ddc5cbaa073b admin admin] PATH_INFO: `/v3/projects/718d72079d934516a5927c2f8f5bf064/users/c0f6c8be888d4c7f81ca6b2244c5614c/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.771443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4380237-891a-4340-b43f-ddc5cbaa073b admin admin] RBAC: Authorizing `identity:create_grant(project_id=718d72079d934516a5927c2f8f5bf064, user_id=c0f6c8be888d4c7f81ca6b2244c5614c, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.774032 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4380237-891a-4340-b43f-ddc5cbaa073b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.784893 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1320/2663] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:22 2022] PUT /identity/v3/projects/718d72079d934516a5927c2f8f5bf064/users/c0f6c8be888d4c7f81ca6b2244c5614c/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:22.795249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a546d8-8c52-4793-a2a6-75d3223abf8e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.795751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a546d8-8c52-4793-a2a6-75d3223abf8e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.795866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a546d8-8c52-4793-a2a6-75d3223abf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.824786 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-83a546d8-8c52-4793-a2a6-75d3223abf8e None None] MFA Rules not processed for user `c0f6c8be888d4c7f81ca6b2244c5614c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:22.826232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-83a546d8-8c52-4793-a2a6-75d3223abf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:22.884763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:22.893918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1344/2664] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:22.911491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4380237-891a-4340-b43f-ddc5cbaa073b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.913263 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a4380237-891a-4340-b43f-ddc5cbaa073b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:22.977197 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4380237-891a-4340-b43f-ddc5cbaa073b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.980930 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c6911-efd2-45bb-ada9-24353ea5f183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.981531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c6911-efd2-45bb-ada9-24353ea5f183 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.981947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c6911-efd2-45bb-ada9-24353ea5f183 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.982221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c6911-efd2-45bb-ada9-24353ea5f183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.991845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c6911-efd2-45bb-ada9-24353ea5f183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.993249 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c6911-efd2-45bb-ada9-24353ea5f183 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:23.002540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1321/2665] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:24.353827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a546d8-8c52-4793-a2a6-75d3223abf8e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:24.414734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a546d8-8c52-4793-a2a6-75d3223abf8e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:24.419524 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:24.420156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:24.420433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:24.420709 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] PATH_INFO: `/v3/users/82232ea9c33f4bf19ad725cfed7c7f4a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:24.421832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] RBAC: Authorizing `identity:delete_user(user_id=82232ea9c33f4bf19ad725cfed7c7f4a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:24.424307 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:24.470315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82232ea9c33f4bf19ad725cfed7c7f4a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:24.470955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82232ea9c33f4bf19ad725cfed7c7f4a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:24.482607 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82232ea9c33f4bf19ad725cfed7c7f4a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:24.490468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82232ea9c33f4bf19ad725cfed7c7f4a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:24.497354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82232ea9c33f4bf19ad725cfed7c7f4a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:24.503188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1345/2666] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:24 2022] DELETE /identity/v3/users/82232ea9c33f4bf19ad725cfed7c7f4a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:24.507729 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2b937-f30d-4707-85f6-981587aa40ff None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:24.508216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2b937-f30d-4707-85f6-981587aa40ff None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:24.508326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2b937-f30d-4707-85f6-981587aa40ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:24.526835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-75d2b937-f30d-4707-85f6-981587aa40ff None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:24.528127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-75d2b937-f30d-4707-85f6-981587aa40ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:24.585902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1322/2667] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:24.596057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:24.597445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:24.652689 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960214e-fa71-4502-bfda-b94c5131f43b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:24.656273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742c334-8257-44c1-bf3a-259e926964ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:24.657410 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742c334-8257-44c1-bf3a-259e926964ec service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:24.657686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742c334-8257-44c1-bf3a-259e926964ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:24.657950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742c334-8257-44c1-bf3a-259e926964ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:24.666693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742c334-8257-44c1-bf3a-259e926964ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:24.668132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742c334-8257-44c1-bf3a-259e926964ec service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:24.677955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1346/2668] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:24.946434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2b937-f30d-4707-85f6-981587aa40ff None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:24.947942 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-75d2b937-f30d-4707-85f6-981587aa40ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:24.962251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2b937-f30d-4707-85f6-981587aa40ff None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:24.966843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:24.967509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:24.967784 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:24.968081 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] PATH_INFO: `/v3/projects/7bfabad318fd45589a834c5593fe7508` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:24.971836 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bfabad318fd45589a834c5593fe7508)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:24.973921 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:25.016798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:25.026717 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bfabad318fd45589a834c5593fe7508', 'request_id': 'req-593cce6f-c792-4ad7-b6fd-228aea8c5a40'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.033742 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bfabad318fd45589a834c5593fe7508', 'request_id': 'req-593cce6f-c792-4ad7-b6fd-228aea8c5a40'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.039366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bfabad318fd45589a834c5593fe7508', 'request_id': 'req-593cce6f-c792-4ad7-b6fd-228aea8c5a40'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.052005 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] The token cache is being invalidate because project 7bfabad318fd45589a834c5593fe7508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:25.052287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.052908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.054387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1323/2669] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:24 2022] DELETE /identity/v3/projects/7bfabad318fd45589a834c5593fe7508 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:25.058522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742c334-8257-44c1-bf3a-259e926964ec service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:25.068789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742c334-8257-44c1-bf3a-259e926964ec service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:25.081182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:25.127508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5267dd-dc3f-4f3a-b229-7fd5cd5d19b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:25.128058 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5267dd-dc3f-4f3a-b229-7fd5cd5d19b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:25.128337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5267dd-dc3f-4f3a-b229-7fd5cd5d19b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:25.128607 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5267dd-dc3f-4f3a-b229-7fd5cd5d19b7 admin admin] PATH_INFO: `/v3/users/c0f6c8be888d4c7f81ca6b2244c5614c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:25.129712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5267dd-dc3f-4f3a-b229-7fd5cd5d19b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0f6c8be888d4c7f81ca6b2244c5614c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:25.131708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5267dd-dc3f-4f3a-b229-7fd5cd5d19b7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:25.177145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3d5267dd-dc3f-4f3a-b229-7fd5cd5d19b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0f6c8be888d4c7f81ca6b2244c5614c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.177808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3d5267dd-dc3f-4f3a-b229-7fd5cd5d19b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0f6c8be888d4c7f81ca6b2244c5614c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.192694 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3d5267dd-dc3f-4f3a-b229-7fd5cd5d19b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0f6c8be888d4c7f81ca6b2244c5614c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.199789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3d5267dd-dc3f-4f3a-b229-7fd5cd5d19b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0f6c8be888d4c7f81ca6b2244c5614c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.207226 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3d5267dd-dc3f-4f3a-b229-7fd5cd5d19b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0f6c8be888d4c7f81ca6b2244c5614c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.212984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1347/2670] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:25 2022] DELETE /identity/v3/users/c0f6c8be888d4c7f81ca6b2244c5614c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:25.483716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:25.485721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:25.499589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cce6f-c792-4ad7-b6fd-228aea8c5a40 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:25.556239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:25.556854 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:25.557125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:25.557396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] PATH_INFO: `/v3/projects/718d72079d934516a5927c2f8f5bf064` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:25.558489 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=718d72079d934516a5927c2f8f5bf064)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:25.560434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:25.604856 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718d72079d934516a5927c2f8f5bf064', 'request_id': 'req-1015f492-2d5d-4391-ac70-d63e103703a6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.611468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718d72079d934516a5927c2f8f5bf064', 'request_id': 'req-1015f492-2d5d-4391-ac70-d63e103703a6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.616948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718d72079d934516a5927c2f8f5bf064', 'request_id': 'req-1015f492-2d5d-4391-ac70-d63e103703a6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.629058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] The token cache is being invalidate because project 718d72079d934516a5927c2f8f5bf064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:25.629345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.629957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1015f492-2d5d-4391-ac70-d63e103703a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:25.631463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1324/2671] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:25 2022] DELETE /identity/v3/projects/718d72079d934516a5927c2f8f5bf064 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:25.636323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636855a8-6e5b-43e4-97cb-362a4d756ff7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:25.636785 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636855a8-6e5b-43e4-97cb-362a4d756ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:25.636881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636855a8-6e5b-43e4-97cb-362a4d756ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:25.656169 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-636855a8-6e5b-43e4-97cb-362a4d756ff7 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:25.657441 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-636855a8-6e5b-43e4-97cb-362a4d756ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:25.722986 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1348/2672] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:25.732085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd43117-dc5a-42f4-897f-ba80af3cbbd3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:25.732454 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd43117-dc5a-42f4-897f-ba80af3cbbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:25.732767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd43117-dc5a-42f4-897f-ba80af3cbbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:25.753432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9dd43117-dc5a-42f4-897f-ba80af3cbbd3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:25.754641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9dd43117-dc5a-42f4-897f-ba80af3cbbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:25.762546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1325/2673] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:25.767080 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636855a8-6e5b-43e4-97cb-362a4d756ff7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:25.776715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636855a8-6e5b-43e4-97cb-362a4d756ff7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:25.781328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a0b20-999c-4401-b958-1ab7de0bee71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:25.781939 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0b20-999c-4401-b958-1ab7de0bee71 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:25.782215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0b20-999c-4401-b958-1ab7de0bee71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:25.782482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0b20-999c-4401-b958-1ab7de0bee71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:25.783140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a0b20-999c-4401-b958-1ab7de0bee71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:25.783457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a0b20-999c-4401-b958-1ab7de0bee71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:25.785003 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a0b20-999c-4401-b958-1ab7de0bee71 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:25.796519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1349/2674] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:25.801160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd43117-dc5a-42f4-897f-ba80af3cbbd3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:25.810597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd43117-dc5a-42f4-897f-ba80af3cbbd3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:25.815965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85f38d-f592-4945-a8c9-072f3d0b0994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:25.816589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85f38d-f592-4945-a8c9-072f3d0b0994 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:25.816838 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85f38d-f592-4945-a8c9-072f3d0b0994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:25.817208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85f38d-f592-4945-a8c9-072f3d0b0994 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:25.817847 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85f38d-f592-4945-a8c9-072f3d0b0994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:25.819739 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85f38d-f592-4945-a8c9-072f3d0b0994 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:25.846691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1326/2675] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:25 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:25.850881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a0b20-999c-4401-b958-1ab7de0bee71 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:25.861572 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a0b20-999c-4401-b958-1ab7de0bee71 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:25.881371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:25.918306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ba004-7b60-46cb-af38-cdb887bb6c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:25.918836 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ba004-7b60-46cb-af38-cdb887bb6c87 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:25.919163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ba004-7b60-46cb-af38-cdb887bb6c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:25.919435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ba004-7b60-46cb-af38-cdb887bb6c87 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:25.920072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ba004-7b60-46cb-af38-cdb887bb6c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:25.921963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ba004-7b60-46cb-af38-cdb887bb6c87 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:25.943152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1350/2676] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:25 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:25.947325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85f38d-f592-4945-a8c9-072f3d0b0994 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:25.957981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85f38d-f592-4945-a8c9-072f3d0b0994 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:25.962607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c6bdb-aea7-4206-94f0-4d61f24c72e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:25.963232 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c6bdb-aea7-4206-94f0-4d61f24c72e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:25.963515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c6bdb-aea7-4206-94f0-4d61f24c72e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:25.963790 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c6bdb-aea7-4206-94f0-4d61f24c72e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:25.964449 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c6bdb-aea7-4206-94f0-4d61f24c72e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:25.966048 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c6bdb-aea7-4206-94f0-4d61f24c72e6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:25.976329 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1327/2677] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:25.980180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ba004-7b60-46cb-af38-cdb887bb6c87 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:25.990843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ba004-7b60-46cb-af38-cdb887bb6c87 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:25.995382 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bffb2-5f36-4836-8f73-e0944d8ed044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:25.995968 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bffb2-5f36-4836-8f73-e0944d8ed044 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:25.996246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bffb2-5f36-4836-8f73-e0944d8ed044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:25.996519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bffb2-5f36-4836-8f73-e0944d8ed044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:25.997228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bffb2-5f36-4836-8f73-e0944d8ed044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:25.998780 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bffb2-5f36-4836-8f73-e0944d8ed044 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:26.009240 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1351/2678] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:14:26.013311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c6bdb-aea7-4206-94f0-4d61f24c72e6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:26.023846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c6bdb-aea7-4206-94f0-4d61f24c72e6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:26.028649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59323873-9b90-4e79-aa6d-0d2559aba62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:26.029297 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59323873-9b90-4e79-aa6d-0d2559aba62a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:26.029576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59323873-9b90-4e79-aa6d-0d2559aba62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:26.029881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59323873-9b90-4e79-aa6d-0d2559aba62a admin admin] PATH_INFO: `/v3/projects/1ad0c146d76e4accb2f365f3faf84c5b/users/4d3af8daf57a4a9b8c6ab12b01dc9275/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:26.042335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:26.045924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59323873-9b90-4e79-aa6d-0d2559aba62a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad0c146d76e4accb2f365f3faf84c5b, user_id=4d3af8daf57a4a9b8c6ab12b01dc9275, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:26.049783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59323873-9b90-4e79-aa6d-0d2559aba62a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:26.063294 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1328/2679] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:26 2022] PUT /identity/v3/projects/1ad0c146d76e4accb2f365f3faf84c5b/users/4d3af8daf57a4a9b8c6ab12b01dc9275/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:26.073595 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600f48f-cf86-4ea6-b715-18de2420d650 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:26.073885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600f48f-cf86-4ea6-b715-18de2420d650 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:26.074160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600f48f-cf86-4ea6-b715-18de2420d650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:26.107777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8600f48f-cf86-4ea6-b715-18de2420d650 None None] MFA Rules not processed for user `4d3af8daf57a4a9b8c6ab12b01dc9275`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:26.109556 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8600f48f-cf86-4ea6-b715-18de2420d650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:26.177400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1352/2680] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:26.181857 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59323873-9b90-4e79-aa6d-0d2559aba62a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:26.245315 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59323873-9b90-4e79-aa6d-0d2559aba62a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:26.249914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767b96e-bbeb-49f7-b166-bb2a71641dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:26.250542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767b96e-bbeb-49f7-b166-bb2a71641dbf admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:26.251373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767b96e-bbeb-49f7-b166-bb2a71641dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:26.251642 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767b96e-bbeb-49f7-b166-bb2a71641dbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:26.252253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767b96e-bbeb-49f7-b166-bb2a71641dbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:26.254139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767b96e-bbeb-49f7-b166-bb2a71641dbf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:26.280394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1329/2681] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:26 2022] POST /identity/v3/projects => generated 393 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:26.285234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600f48f-cf86-4ea6-b715-18de2420d650 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:26.295885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600f48f-cf86-4ea6-b715-18de2420d650 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:26.353076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d76b7c-264a-4a81-a798-7bc7eeaf7571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:26.353676 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d76b7c-264a-4a81-a798-7bc7eeaf7571 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:26.353955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d76b7c-264a-4a81-a798-7bc7eeaf7571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:26.354237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d76b7c-264a-4a81-a798-7bc7eeaf7571 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:26.354841 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d76b7c-264a-4a81-a798-7bc7eeaf7571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:26.357047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d76b7c-264a-4a81-a798-7bc7eeaf7571 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:26.378064 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1353/2682] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:26 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:26.382139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767b96e-bbeb-49f7-b166-bb2a71641dbf admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:26.393203 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767b96e-bbeb-49f7-b166-bb2a71641dbf admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:26.397938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2924d90-f24b-4cc7-9368-40552a8133f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:26.398561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2924d90-f24b-4cc7-9368-40552a8133f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:26.398932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2924d90-f24b-4cc7-9368-40552a8133f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:26.399238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2924d90-f24b-4cc7-9368-40552a8133f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:26.399880 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2924d90-f24b-4cc7-9368-40552a8133f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:26.401529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2924d90-f24b-4cc7-9368-40552a8133f4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:26.412758 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1330/2683] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:26.417055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d76b7c-264a-4a81-a798-7bc7eeaf7571 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:26.427287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d76b7c-264a-4a81-a798-7bc7eeaf7571 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:26.432597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff136410-9b44-48fb-b808-e6cdd9e83282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:26.433630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff136410-9b44-48fb-b808-e6cdd9e83282 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:26.434121 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff136410-9b44-48fb-b808-e6cdd9e83282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:26.434614 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff136410-9b44-48fb-b808-e6cdd9e83282 admin admin] PATH_INFO: `/v3/projects/aa3d725efa034b0088645e1475827b9a/users/dc1764e196b6415bab38f599dd274dfc/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:26.453047 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff136410-9b44-48fb-b808-e6cdd9e83282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3d725efa034b0088645e1475827b9a, user_id=dc1764e196b6415bab38f599dd274dfc, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:26.456326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff136410-9b44-48fb-b808-e6cdd9e83282 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:26.464440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:26.471210 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1354/2684] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:26 2022] PUT /identity/v3/projects/aa3d725efa034b0088645e1475827b9a/users/dc1764e196b6415bab38f599dd274dfc/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:26.475322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2924d90-f24b-4cc7-9368-40552a8133f4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:26.485681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2924d90-f24b-4cc7-9368-40552a8133f4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:26.542261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d2849-8452-48f0-8cf5-9011464dd771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:26.542865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2849-8452-48f0-8cf5-9011464dd771 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:26.543170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2849-8452-48f0-8cf5-9011464dd771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:26.543442 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2849-8452-48f0-8cf5-9011464dd771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:26.544070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d2849-8452-48f0-8cf5-9011464dd771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:26.545636 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d2849-8452-48f0-8cf5-9011464dd771 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:26.555261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1331/2685] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:26.559508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff136410-9b44-48fb-b808-e6cdd9e83282 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:26.569945 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff136410-9b44-48fb-b808-e6cdd9e83282 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:26.574503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3df44-3ac0-4e32-a2f9-7c7c33e0b020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:26.575203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3df44-3ac0-4e32-a2f9-7c7c33e0b020 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:26.575485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3df44-3ac0-4e32-a2f9-7c7c33e0b020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:26.575758 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3df44-3ac0-4e32-a2f9-7c7c33e0b020 admin admin] PATH_INFO: `/v3/projects/aa3d725efa034b0088645e1475827b9a/users/dc1764e196b6415bab38f599dd274dfc/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:26.577942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3df44-3ac0-4e32-a2f9-7c7c33e0b020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3d725efa034b0088645e1475827b9a, user_id=dc1764e196b6415bab38f599dd274dfc, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:26.581107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3df44-3ac0-4e32-a2f9-7c7c33e0b020 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:26.591246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1355/2686] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:26 2022] PUT /identity/v3/projects/aa3d725efa034b0088645e1475827b9a/users/dc1764e196b6415bab38f599dd274dfc/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:26.602375 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155ff9a-8931-46d0-bc48-76f19409679d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:26.602862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155ff9a-8931-46d0-bc48-76f19409679d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:26.602968 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155ff9a-8931-46d0-bc48-76f19409679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:26.630833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b155ff9a-8931-46d0-bc48-76f19409679d None None] MFA Rules not processed for user `dc1764e196b6415bab38f599dd274dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:26.632099 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b155ff9a-8931-46d0-bc48-76f19409679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:26.679321 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:26.701607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1332/2687] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:26 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:26.713283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3df44-3ac0-4e32-a2f9-7c7c33e0b020 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:26.715071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8de3df44-3ac0-4e32-a2f9-7c7c33e0b020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:26.772542 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3df44-3ac0-4e32-a2f9-7c7c33e0b020 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:26.776172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa2227-fe46-4a11-8b65-d9470d6d6494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:26.776770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa2227-fe46-4a11-8b65-d9470d6d6494 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:26.777053 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa2227-fe46-4a11-8b65-d9470d6d6494 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:26.777328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa2227-fe46-4a11-8b65-d9470d6d6494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:26.786357 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa2227-fe46-4a11-8b65-d9470d6d6494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:26.787812 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa2227-fe46-4a11-8b65-d9470d6d6494 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:26.797434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1356/2688] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:26 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:26.845741 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155ff9a-8931-46d0-bc48-76f19409679d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:26.847168 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b155ff9a-8931-46d0-bc48-76f19409679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:26.859546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155ff9a-8931-46d0-bc48-76f19409679d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:26.863225 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e2757-220b-466d-bba5-fa8be947c48e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:26.863853 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e2757-220b-466d-bba5-fa8be947c48e service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:26.864135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e2757-220b-466d-bba5-fa8be947c48e service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:26.864408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e2757-220b-466d-bba5-fa8be947c48e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:26.920144 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e2757-220b-466d-bba5-fa8be947c48e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:26.921563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e2757-220b-466d-bba5-fa8be947c48e service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:26.930464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1333/2689] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:30.197468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa2227-fe46-4a11-8b65-d9470d6d6494 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:30.261533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa2227-fe46-4a11-8b65-d9470d6d6494 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.268584 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.269223 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.269498 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.269764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] PATH_INFO: `/v3/users/4d3af8daf57a4a9b8c6ab12b01dc9275` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.270907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d3af8daf57a4a9b8c6ab12b01dc9275)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.272829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:30.317312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d3af8daf57a4a9b8c6ab12b01dc9275'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.317916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d3af8daf57a4a9b8c6ab12b01dc9275'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.332090 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d3af8daf57a4a9b8c6ab12b01dc9275'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.339509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d3af8daf57a4a9b8c6ab12b01dc9275'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.347187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d3af8daf57a4a9b8c6ab12b01dc9275'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.352972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1357/2690] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:30 2022] DELETE /identity/v3/users/4d3af8daf57a4a9b8c6ab12b01dc9275 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:30.357667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05786e2-fca4-4500-84d1-3e5549493aef None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.358126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05786e2-fca4-4500-84d1-3e5549493aef None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.358221 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05786e2-fca4-4500-84d1-3e5549493aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.377683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b05786e2-fca4-4500-84d1-3e5549493aef None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:30.378883 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b05786e2-fca4-4500-84d1-3e5549493aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:30.436858 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1334/2691] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:30.447133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:30.448558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:30.459192 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:30.506862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17474275-7683-4ad1-966e-129989fefd59 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.510655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fafd3-01fb-451e-b90d-576af774cc32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.511310 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fafd3-01fb-451e-b90d-576af774cc32 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.511593 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fafd3-01fb-451e-b90d-576af774cc32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.511868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fafd3-01fb-451e-b90d-576af774cc32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.521681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fafd3-01fb-451e-b90d-576af774cc32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.523146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fafd3-01fb-451e-b90d-576af774cc32 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:30.532869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1358/2692] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:30.805938 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05786e2-fca4-4500-84d1-3e5549493aef None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:30.810896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b05786e2-fca4-4500-84d1-3e5549493aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:30.821518 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05786e2-fca4-4500-84d1-3e5549493aef None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.826504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.827163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.827443 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.827712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] PATH_INFO: `/v3/projects/1ad0c146d76e4accb2f365f3faf84c5b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.828834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ad0c146d76e4accb2f365f3faf84c5b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.830705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:30.876293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ad0c146d76e4accb2f365f3faf84c5b', 'request_id': 'req-42a715e2-388d-4ad4-a5f4-002834eb4d31'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.882248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ad0c146d76e4accb2f365f3faf84c5b', 'request_id': 'req-42a715e2-388d-4ad4-a5f4-002834eb4d31'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.887789 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ad0c146d76e4accb2f365f3faf84c5b', 'request_id': 'req-42a715e2-388d-4ad4-a5f4-002834eb4d31'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.900305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] The token cache is being invalidate because project 1ad0c146d76e4accb2f365f3faf84c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:30.900596 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.901243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.902802 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1335/2693] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:30 2022] DELETE /identity/v3/projects/1ad0c146d76e4accb2f365f3faf84c5b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:30.907247 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fafd3-01fb-451e-b90d-576af774cc32 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:30.917656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fafd3-01fb-451e-b90d-576af774cc32 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.973181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59f3b5-7d95-4475-9361-fb9176a04371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.973808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59f3b5-7d95-4475-9361-fb9176a04371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.974088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59f3b5-7d95-4475-9361-fb9176a04371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.974362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59f3b5-7d95-4475-9361-fb9176a04371 admin admin] PATH_INFO: `/v3/users/dc1764e196b6415bab38f599dd274dfc` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.975508 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59f3b5-7d95-4475-9361-fb9176a04371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc1764e196b6415bab38f599dd274dfc)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.977454 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59f3b5-7d95-4475-9361-fb9176a04371 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:31.023656 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e59f3b5-7d95-4475-9361-fb9176a04371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc1764e196b6415bab38f599dd274dfc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:31.024298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e59f3b5-7d95-4475-9361-fb9176a04371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc1764e196b6415bab38f599dd274dfc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:31.035991 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e59f3b5-7d95-4475-9361-fb9176a04371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc1764e196b6415bab38f599dd274dfc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:31.044538 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e59f3b5-7d95-4475-9361-fb9176a04371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc1764e196b6415bab38f599dd274dfc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:31.052761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e59f3b5-7d95-4475-9361-fb9176a04371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc1764e196b6415bab38f599dd274dfc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:31.058745 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1359/2694] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:30 2022] DELETE /identity/v3/users/dc1764e196b6415bab38f599dd274dfc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:31.606882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:31.608622 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:31.625160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a715e2-388d-4ad4-a5f4-002834eb4d31 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:31.631307 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:31.685361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:31.686133 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:31.686563 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:31.686865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] PATH_INFO: `/v3/projects/aa3d725efa034b0088645e1475827b9a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:31.688078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3d725efa034b0088645e1475827b9a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:31.689978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:31.737127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3d725efa034b0088645e1475827b9a', 'request_id': 'req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:31.743169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3d725efa034b0088645e1475827b9a', 'request_id': 'req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:31.748735 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3d725efa034b0088645e1475827b9a', 'request_id': 'req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:31.760874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] The token cache is being invalidate because project aa3d725efa034b0088645e1475827b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:31.761163 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:31.761777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2e6475a8-7cf1-49df-9ca3-ee4bf5372802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:31.763516 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1336/2695] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:31 2022] DELETE /identity/v3/projects/aa3d725efa034b0088645e1475827b9a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:31.768960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4c283-3e3e-474d-892e-526aa068ea49 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:31.769448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4c283-3e3e-474d-892e-526aa068ea49 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:31.769549 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4c283-3e3e-474d-892e-526aa068ea49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:31.789455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b5c4c283-3e3e-474d-892e-526aa068ea49 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:31.790639 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b5c4c283-3e3e-474d-892e-526aa068ea49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:31.814314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:31.852273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1360/2696] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:31.861033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800b979-4818-4355-9d8b-4b6d158fc22f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:31.861345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800b979-4818-4355-9d8b-4b6d158fc22f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:31.861620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800b979-4818-4355-9d8b-4b6d158fc22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:31.881800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d800b979-4818-4355-9d8b-4b6d158fc22f None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:31.883022 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d800b979-4818-4355-9d8b-4b6d158fc22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:31.891497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1337/2697] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:31.895734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4c283-3e3e-474d-892e-526aa068ea49 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:31.905694 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4c283-3e3e-474d-892e-526aa068ea49 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:31.910145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c4d22-365d-4edf-af58-cd4a906ec61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:31.910738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c4d22-365d-4edf-af58-cd4a906ec61d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:31.911038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c4d22-365d-4edf-af58-cd4a906ec61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:31.911314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c4d22-365d-4edf-af58-cd4a906ec61d admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:31.911952 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c4d22-365d-4edf-af58-cd4a906ec61d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:31.912252 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c4d22-365d-4edf-af58-cd4a906ec61d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:31.913846 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c4d22-365d-4edf-af58-cd4a906ec61d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:31.925287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1361/2698] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:31.930230 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800b979-4818-4355-9d8b-4b6d158fc22f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:31.940707 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800b979-4818-4355-9d8b-4b6d158fc22f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:31.945959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f095bd-d296-4315-9aa4-73b283cf0f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:31.946570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f095bd-d296-4315-9aa4-73b283cf0f97 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:31.946970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f095bd-d296-4315-9aa4-73b283cf0f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:31.947306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f095bd-d296-4315-9aa4-73b283cf0f97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:31.948045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f095bd-d296-4315-9aa4-73b283cf0f97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:31.950087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f095bd-d296-4315-9aa4-73b283cf0f97 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:31.978655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1338/2699] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:31 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:31.982927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c4d22-365d-4edf-af58-cd4a906ec61d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:31.993062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c4d22-365d-4edf-af58-cd4a906ec61d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:32.049653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92807842-aead-4ea2-b00c-113cb11ac60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:32.050252 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92807842-aead-4ea2-b00c-113cb11ac60e admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.050532 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92807842-aead-4ea2-b00c-113cb11ac60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.050805 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92807842-aead-4ea2-b00c-113cb11ac60e admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.051471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92807842-aead-4ea2-b00c-113cb11ac60e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:32.053373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92807842-aead-4ea2-b00c-113cb11ac60e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:32.073674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1362/2700] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:31 2022] POST /identity/v3/users => generated 301 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:32.078095 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f095bd-d296-4315-9aa4-73b283cf0f97 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:32.088951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f095bd-d296-4315-9aa4-73b283cf0f97 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:32.093543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfbe16-2cab-4983-bca8-4bd69b300235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:32.094136 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfbe16-2cab-4983-bca8-4bd69b300235 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.094529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfbe16-2cab-4983-bca8-4bd69b300235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.094807 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfbe16-2cab-4983-bca8-4bd69b300235 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.095477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfbe16-2cab-4983-bca8-4bd69b300235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:32.097066 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfbe16-2cab-4983-bca8-4bd69b300235 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:32.106798 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1339/2701] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:32.110728 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92807842-aead-4ea2-b00c-113cb11ac60e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:32.121334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92807842-aead-4ea2-b00c-113cb11ac60e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:32.126370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f34365-057c-47a8-98e7-d1a7ed857d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:32.126953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f34365-057c-47a8-98e7-d1a7ed857d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.127254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f34365-057c-47a8-98e7-d1a7ed857d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.127521 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f34365-057c-47a8-98e7-d1a7ed857d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.128175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f34365-057c-47a8-98e7-d1a7ed857d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:32.129757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f34365-057c-47a8-98e7-d1a7ed857d94 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:32.141636 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1363/2702] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:32.146006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfbe16-2cab-4983-bca8-4bd69b300235 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:32.156553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfbe16-2cab-4983-bca8-4bd69b300235 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:32.161260 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2993fd-b089-4e15-b5b7-86cfbe18fc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:32.161907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2993fd-b089-4e15-b5b7-86cfbe18fc32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.162190 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2993fd-b089-4e15-b5b7-86cfbe18fc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.162469 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2993fd-b089-4e15-b5b7-86cfbe18fc32 admin admin] PATH_INFO: `/v3/projects/149b1b4108304635b8d1dd43da7d9e6e/users/09b8037c60dd42448cab95240f680483/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.176878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2993fd-b089-4e15-b5b7-86cfbe18fc32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=149b1b4108304635b8d1dd43da7d9e6e, user_id=09b8037c60dd42448cab95240f680483, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:32.179559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2993fd-b089-4e15-b5b7-86cfbe18fc32 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:32.195407 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1340/2703] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:32 2022] PUT /identity/v3/projects/149b1b4108304635b8d1dd43da7d9e6e/users/09b8037c60dd42448cab95240f680483/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:32.206637 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e02ef-2c10-46c5-8636-b808be3d2ff9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.206921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e02ef-2c10-46c5-8636-b808be3d2ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.207237 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e02ef-2c10-46c5-8636-b808be3d2ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.235454 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-144e02ef-2c10-46c5-8636-b808be3d2ff9 None None] MFA Rules not processed for user `09b8037c60dd42448cab95240f680483`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:32.236697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-144e02ef-2c10-46c5-8636-b808be3d2ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:32.302242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1364/2704] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:32 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:32.306873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2993fd-b089-4e15-b5b7-86cfbe18fc32 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:32.347548 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:32.372373 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2993fd-b089-4e15-b5b7-86cfbe18fc32 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:32.377514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c6dd7-316a-4af4-9d63-7d57948769f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:32.378106 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c6dd7-316a-4af4-9d63-7d57948769f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.378383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c6dd7-316a-4af4-9d63-7d57948769f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.378652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c6dd7-316a-4af4-9d63-7d57948769f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.379305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c6dd7-316a-4af4-9d63-7d57948769f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:32.381179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c6dd7-316a-4af4-9d63-7d57948769f4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:32.406691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1341/2705] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:32 2022] POST /identity/v3/projects => generated 399 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:32.411165 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e02ef-2c10-46c5-8636-b808be3d2ff9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:32.421768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e02ef-2c10-46c5-8636-b808be3d2ff9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:32.442380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:32.479938 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a0fbf-6099-4d54-b687-aacbcb682df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:32.480541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a0fbf-6099-4d54-b687-aacbcb682df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.480876 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a0fbf-6099-4d54-b687-aacbcb682df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.481187 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a0fbf-6099-4d54-b687-aacbcb682df1 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.481700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a0fbf-6099-4d54-b687-aacbcb682df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:32.483645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a0fbf-6099-4d54-b687-aacbcb682df1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:32.505722 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1365/2706] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:32 2022] POST /identity/v3/users => generated 306 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:32.509963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c6dd7-316a-4af4-9d63-7d57948769f4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:32.520192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c6dd7-316a-4af4-9d63-7d57948769f4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:32.524767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb8bb3-cd3f-4103-a8ec-b30830bd9f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:32.525354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb8bb3-cd3f-4103-a8ec-b30830bd9f45 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.525632 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb8bb3-cd3f-4103-a8ec-b30830bd9f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.525901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb8bb3-cd3f-4103-a8ec-b30830bd9f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.526529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb8bb3-cd3f-4103-a8ec-b30830bd9f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:32.528254 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb8bb3-cd3f-4103-a8ec-b30830bd9f45 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:32.661577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1342/2707] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:32.665678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a0fbf-6099-4d54-b687-aacbcb682df1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:32.676471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a0fbf-6099-4d54-b687-aacbcb682df1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:32.681132 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681d368-5d6c-4f6f-9886-25a65b7681f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:32.681787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681d368-5d6c-4f6f-9886-25a65b7681f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.682068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681d368-5d6c-4f6f-9886-25a65b7681f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.682343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681d368-5d6c-4f6f-9886-25a65b7681f8 admin admin] PATH_INFO: `/v3/projects/de79e10b16334469b322c870917d56c8/users/0928876d36584cb0b9b30103d931f243/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.697643 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681d368-5d6c-4f6f-9886-25a65b7681f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de79e10b16334469b322c870917d56c8, user_id=0928876d36584cb0b9b30103d931f243, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:32.700346 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681d368-5d6c-4f6f-9886-25a65b7681f8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:32.713510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1366/2708] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:32 2022] PUT /identity/v3/projects/de79e10b16334469b322c870917d56c8/users/0928876d36584cb0b9b30103d931f243/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:32.717842 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb8bb3-cd3f-4103-a8ec-b30830bd9f45 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:32.728162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb8bb3-cd3f-4103-a8ec-b30830bd9f45 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:32.787214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d841d-a6b6-432a-88c5-46c9fdb3f12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:32.787822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d841d-a6b6-432a-88c5-46c9fdb3f12e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.788098 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d841d-a6b6-432a-88c5-46c9fdb3f12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.788369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d841d-a6b6-432a-88c5-46c9fdb3f12e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.788997 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d841d-a6b6-432a-88c5-46c9fdb3f12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:32.790602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d841d-a6b6-432a-88c5-46c9fdb3f12e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:32.800976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1343/2709] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:14:32.805502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681d368-5d6c-4f6f-9886-25a65b7681f8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:32.816040 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681d368-5d6c-4f6f-9886-25a65b7681f8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:32.820835 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723c876-a0cd-4e91-81c7-aebba3fe01dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:32.821489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723c876-a0cd-4e91-81c7-aebba3fe01dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.821768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723c876-a0cd-4e91-81c7-aebba3fe01dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.822038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723c876-a0cd-4e91-81c7-aebba3fe01dc admin admin] PATH_INFO: `/v3/projects/de79e10b16334469b322c870917d56c8/users/0928876d36584cb0b9b30103d931f243/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.823989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723c876-a0cd-4e91-81c7-aebba3fe01dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=de79e10b16334469b322c870917d56c8, user_id=0928876d36584cb0b9b30103d931f243, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:32.826946 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723c876-a0cd-4e91-81c7-aebba3fe01dc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:32.837086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1367/2710] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:32 2022] PUT /identity/v3/projects/de79e10b16334469b322c870917d56c8/users/0928876d36584cb0b9b30103d931f243/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:32.848403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c134b0-e74e-49de-a8a0-8e9161db6a70 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:32.848693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c134b0-e74e-49de-a8a0-8e9161db6a70 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:32.848970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c134b0-e74e-49de-a8a0-8e9161db6a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:32.876301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-25c134b0-e74e-49de-a8a0-8e9161db6a70 None None] MFA Rules not processed for user `0928876d36584cb0b9b30103d931f243`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:32.877508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-25c134b0-e74e-49de-a8a0-8e9161db6a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:32.945768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1344/2711] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:32 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:32.957354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723c876-a0cd-4e91-81c7-aebba3fe01dc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:32.959021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d723c876-a0cd-4e91-81c7-aebba3fe01dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:33.017041 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723c876-a0cd-4e91-81c7-aebba3fe01dc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:33.020707 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8ea16-cbb7-4596-8ac0-0d0087dbe67a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:33.021302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8ea16-cbb7-4596-8ac0-0d0087dbe67a service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:33.021581 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8ea16-cbb7-4596-8ac0-0d0087dbe67a service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:33.021852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8ea16-cbb7-4596-8ac0-0d0087dbe67a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:33.031862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8ea16-cbb7-4596-8ac0-0d0087dbe67a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:33.033283 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8ea16-cbb7-4596-8ac0-0d0087dbe67a service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:33.042516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1368/2712] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:32 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:33.118463 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c134b0-e74e-49de-a8a0-8e9161db6a70 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:33.128760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c134b0-e74e-49de-a8a0-8e9161db6a70 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:33.132348 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf6b9c-6799-42da-b86d-2311e3dbdf33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:33.132943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf6b9c-6799-42da-b86d-2311e3dbdf33 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:33.133220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf6b9c-6799-42da-b86d-2311e3dbdf33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:33.133497 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf6b9c-6799-42da-b86d-2311e3dbdf33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:33.161616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:33.191161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf6b9c-6799-42da-b86d-2311e3dbdf33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:33.192581 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf6b9c-6799-42da-b86d-2311e3dbdf33 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:33.201668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1345/2713] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:33 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:36.967426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8ea16-cbb7-4596-8ac0-0d0087dbe67a service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:36.968895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6db8ea16-cbb7-4596-8ac0-0d0087dbe67a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:36.980789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8ea16-cbb7-4596-8ac0-0d0087dbe67a service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:36.984332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796d2ce-cae0-4311-88c9-5ef97b16b1ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:36.984914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796d2ce-cae0-4311-88c9-5ef97b16b1ad service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:36.985189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796d2ce-cae0-4311-88c9-5ef97b16b1ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:36.985456 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796d2ce-cae0-4311-88c9-5ef97b16b1ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:36.987115 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d796d2ce-cae0-4311-88c9-5ef97b16b1ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:37.044881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796d2ce-cae0-4311-88c9-5ef97b16b1ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:37.046303 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796d2ce-cae0-4311-88c9-5ef97b16b1ad service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:37.056027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1369/2714] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:37.257757 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf6b9c-6799-42da-b86d-2311e3dbdf33 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:37.324006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf6b9c-6799-42da-b86d-2311e3dbdf33 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:37.328849 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:37.329503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:37.329777 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:37.330044 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] PATH_INFO: `/v3/users/09b8037c60dd42448cab95240f680483` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:37.331713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b8037c60dd42448cab95240f680483)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:37.333602 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:37.378886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b8037c60dd42448cab95240f680483'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:37.390758 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b8037c60dd42448cab95240f680483'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:37.391459 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b8037c60dd42448cab95240f680483'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:37.392668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b8037c60dd42448cab95240f680483'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:37.399963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b8037c60dd42448cab95240f680483'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:37.411881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1346/2715] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:37 2022] DELETE /identity/v3/users/09b8037c60dd42448cab95240f680483 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:37.416370 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ed892-a24c-41e8-bee7-c8d750d39d6b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:37.416659 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ed892-a24c-41e8-bee7-c8d750d39d6b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:37.416940 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ed892-a24c-41e8-bee7-c8d750d39d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:37.436492 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3a4ed892-a24c-41e8-bee7-c8d750d39d6b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:37.437755 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3a4ed892-a24c-41e8-bee7-c8d750d39d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:37.450226 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:37.499141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1370/2716] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:37.508517 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:37.509913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:37.562133 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:37.566335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328338d1-4c68-4a6c-906c-9a7f2ab96d07 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:37.571015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc70ac-fb25-429a-b4eb-46ccb09d345d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:37.571773 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc70ac-fb25-429a-b4eb-46ccb09d345d service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:37.572178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc70ac-fb25-429a-b4eb-46ccb09d345d service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:37.572612 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc70ac-fb25-429a-b4eb-46ccb09d345d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:37.582515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc70ac-fb25-429a-b4eb-46ccb09d345d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:37.583948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc70ac-fb25-429a-b4eb-46ccb09d345d service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:37.594592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1347/2717] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:37.859257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ed892-a24c-41e8-bee7-c8d750d39d6b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:37.860660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3a4ed892-a24c-41e8-bee7-c8d750d39d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:37.874677 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ed892-a24c-41e8-bee7-c8d750d39d6b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:37.879902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:37.881232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:37.881517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:37.881829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] PATH_INFO: `/v3/projects/149b1b4108304635b8d1dd43da7d9e6e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:37.883086 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=149b1b4108304635b8d1dd43da7d9e6e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:37.885208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:37.931638 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149b1b4108304635b8d1dd43da7d9e6e', 'request_id': 'req-9d280f7c-a9c3-4647-a882-e3db9f372ba0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:37.939872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149b1b4108304635b8d1dd43da7d9e6e', 'request_id': 'req-9d280f7c-a9c3-4647-a882-e3db9f372ba0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:37.945935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149b1b4108304635b8d1dd43da7d9e6e', 'request_id': 'req-9d280f7c-a9c3-4647-a882-e3db9f372ba0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:37.957249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] The token cache is being invalidate because project 149b1b4108304635b8d1dd43da7d9e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:37.957541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:37.958172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:37.959672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1371/2718] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:37 2022] DELETE /identity/v3/projects/149b1b4108304635b8d1dd43da7d9e6e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:37.963742 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc70ac-fb25-429a-b4eb-46ccb09d345d service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:37.973623 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc70ac-fb25-429a-b4eb-46ccb09d345d service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:38.029765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af4c35-00cb-4df5-a3f6-74083508fca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:38.030388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af4c35-00cb-4df5-a3f6-74083508fca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.030669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af4c35-00cb-4df5-a3f6-74083508fca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.031085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af4c35-00cb-4df5-a3f6-74083508fca9 admin admin] PATH_INFO: `/v3/users/0928876d36584cb0b9b30103d931f243` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.032264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af4c35-00cb-4df5-a3f6-74083508fca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0928876d36584cb0b9b30103d931f243)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:38.034424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af4c35-00cb-4df5-a3f6-74083508fca9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:38.082495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-12af4c35-00cb-4df5-a3f6-74083508fca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0928876d36584cb0b9b30103d931f243'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:38.094273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-12af4c35-00cb-4df5-a3f6-74083508fca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0928876d36584cb0b9b30103d931f243'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:38.094971 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-12af4c35-00cb-4df5-a3f6-74083508fca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0928876d36584cb0b9b30103d931f243'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:38.095603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-12af4c35-00cb-4df5-a3f6-74083508fca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0928876d36584cb0b9b30103d931f243'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:38.102617 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-12af4c35-00cb-4df5-a3f6-74083508fca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0928876d36584cb0b9b30103d931f243'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:38.114961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1348/2719] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:37 2022] DELETE /identity/v3/users/0928876d36584cb0b9b30103d931f243 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:38.395661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:38.397250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:38.412120 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d280f7c-a9c3-4647-a882-e3db9f372ba0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:38.478470 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:38.479141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.479425 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.479701 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] PATH_INFO: `/v3/projects/de79e10b16334469b322c870917d56c8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.480810 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=de79e10b16334469b322c870917d56c8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:38.482709 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:38.525878 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:38.529908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de79e10b16334469b322c870917d56c8', 'request_id': 'req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:38.537205 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de79e10b16334469b322c870917d56c8', 'request_id': 'req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:38.543350 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de79e10b16334469b322c870917d56c8', 'request_id': 'req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:38.554032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] The token cache is being invalidate because project de79e10b16334469b322c870917d56c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:38.554323 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:38.554950 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c9d7256d-091d-44a5-86a5-78e23c7ddb2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:38.556460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1372/2720] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:38 2022] DELETE /identity/v3/projects/de79e10b16334469b322c870917d56c8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:38.562336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e289a15-0a9c-44a9-a4a8-689b2b1141c5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.562823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e289a15-0a9c-44a9-a4a8-689b2b1141c5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.562937 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e289a15-0a9c-44a9-a4a8-689b2b1141c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.582887 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2e289a15-0a9c-44a9-a4a8-689b2b1141c5 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:38.584128 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2e289a15-0a9c-44a9-a4a8-689b2b1141c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:38.650492 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1349/2721] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:38.659380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951aa72-f18b-4130-825d-8116b11ccff6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.659664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951aa72-f18b-4130-825d-8116b11ccff6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.659942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951aa72-f18b-4130-825d-8116b11ccff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.681488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d951aa72-f18b-4130-825d-8116b11ccff6 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:38.682800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d951aa72-f18b-4130-825d-8116b11ccff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:38.691381 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1373/2722] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:38.695933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e289a15-0a9c-44a9-a4a8-689b2b1141c5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:38.706586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e289a15-0a9c-44a9-a4a8-689b2b1141c5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:38.711192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c6546-ed7d-40eb-987b-aee14b97f6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:38.711792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c6546-ed7d-40eb-987b-aee14b97f6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.712071 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c6546-ed7d-40eb-987b-aee14b97f6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.712345 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c6546-ed7d-40eb-987b-aee14b97f6f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.713008 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c6546-ed7d-40eb-987b-aee14b97f6f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:38.713311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c6546-ed7d-40eb-987b-aee14b97f6f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:38.714979 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c6546-ed7d-40eb-987b-aee14b97f6f4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:38.726424 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:38.732231 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1350/2723] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:38.737250 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951aa72-f18b-4130-825d-8116b11ccff6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:38.746504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951aa72-f18b-4130-825d-8116b11ccff6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:38.751200 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f631d1a-5a3b-4ce1-817a-7bb85a4acaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:38.751794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f631d1a-5a3b-4ce1-817a-7bb85a4acaa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.752069 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f631d1a-5a3b-4ce1-817a-7bb85a4acaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.752341 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f631d1a-5a3b-4ce1-817a-7bb85a4acaa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.752964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f631d1a-5a3b-4ce1-817a-7bb85a4acaa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:38.754872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f631d1a-5a3b-4ce1-817a-7bb85a4acaa8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:38.781213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1374/2724] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:38 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:38.785803 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c6546-ed7d-40eb-987b-aee14b97f6f4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:38.795647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c6546-ed7d-40eb-987b-aee14b97f6f4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:38.853224 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd9d8b-42da-4821-911e-d1690f8626ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:38.853831 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd9d8b-42da-4821-911e-d1690f8626ca admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.854115 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd9d8b-42da-4821-911e-d1690f8626ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.854393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd9d8b-42da-4821-911e-d1690f8626ca admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.855034 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd9d8b-42da-4821-911e-d1690f8626ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:38.856933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd9d8b-42da-4821-911e-d1690f8626ca admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:38.878103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1351/2725] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:38 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:38.882265 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f631d1a-5a3b-4ce1-817a-7bb85a4acaa8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:38.892611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f631d1a-5a3b-4ce1-817a-7bb85a4acaa8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:38.897229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3846b87-7845-4d8a-babd-a050bda3d5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:38.897819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3846b87-7845-4d8a-babd-a050bda3d5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.898127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3846b87-7845-4d8a-babd-a050bda3d5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.898402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3846b87-7845-4d8a-babd-a050bda3d5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.899068 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3846b87-7845-4d8a-babd-a050bda3d5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:38.900761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3846b87-7845-4d8a-babd-a050bda3d5e9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:38.911967 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1375/2726] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:38.915835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd9d8b-42da-4821-911e-d1690f8626ca admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:38.926890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd9d8b-42da-4821-911e-d1690f8626ca admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:38.931852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafeb46e-7c3c-433a-b0e1-7810b193d149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:38.932451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeb46e-7c3c-433a-b0e1-7810b193d149 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.932755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeb46e-7c3c-433a-b0e1-7810b193d149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.933032 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeb46e-7c3c-433a-b0e1-7810b193d149 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.933672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafeb46e-7c3c-433a-b0e1-7810b193d149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:38.935311 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafeb46e-7c3c-433a-b0e1-7810b193d149 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:38.945834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1352/2727] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:38.950088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3846b87-7845-4d8a-babd-a050bda3d5e9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:38.960993 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3846b87-7845-4d8a-babd-a050bda3d5e9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:38.965793 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3886b-42d9-4bfc-b919-96d6e0e2865c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:38.966434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3886b-42d9-4bfc-b919-96d6e0e2865c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.966716 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3886b-42d9-4bfc-b919-96d6e0e2865c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.967013 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3886b-42d9-4bfc-b919-96d6e0e2865c admin admin] PATH_INFO: `/v3/projects/016678f8650943758f2f5f9dc285bf7a/users/fb2dbb409fac421d91e9d7bf0b583ecb/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.983947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3886b-42d9-4bfc-b919-96d6e0e2865c admin admin] RBAC: Authorizing `identity:create_grant(project_id=016678f8650943758f2f5f9dc285bf7a, user_id=fb2dbb409fac421d91e9d7bf0b583ecb, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:38.986683 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3886b-42d9-4bfc-b919-96d6e0e2865c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:39.012568 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1376/2728] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:38 2022] PUT /identity/v3/projects/016678f8650943758f2f5f9dc285bf7a/users/fb2dbb409fac421d91e9d7bf0b583ecb/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:39.024264 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699162a-5cde-4c0f-bc5a-a7d09f746275 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:39.024555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699162a-5cde-4c0f-bc5a-a7d09f746275 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:39.024885 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699162a-5cde-4c0f-bc5a-a7d09f746275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:39.054941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a699162a-5cde-4c0f-bc5a-a7d09f746275 None None] MFA Rules not processed for user `fb2dbb409fac421d91e9d7bf0b583ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:39.056187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a699162a-5cde-4c0f-bc5a-a7d09f746275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:39.120031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1353/2729] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:39.131327 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3886b-42d9-4bfc-b919-96d6e0e2865c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:39.133018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e7c3886b-42d9-4bfc-b919-96d6e0e2865c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:39.190861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3886b-42d9-4bfc-b919-96d6e0e2865c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:39.194477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21dfa2-8c78-425f-88b4-29b97c3e488d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:39.195114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21dfa2-8c78-425f-88b4-29b97c3e488d service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:39.195400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21dfa2-8c78-425f-88b4-29b97c3e488d service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:39.195674 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21dfa2-8c78-425f-88b4-29b97c3e488d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:39.204957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21dfa2-8c78-425f-88b4-29b97c3e488d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:39.206380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21dfa2-8c78-425f-88b4-29b97c3e488d service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:39.215287 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1377/2730] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:49.117050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:49.157845 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c014ef-b625-40ad-b5cc-fa72db0c2332 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:49.158127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c014ef-b625-40ad-b5cc-fa72db0c2332 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:49.158398 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c014ef-b625-40ad-b5cc-fa72db0c2332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:49.176796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-81c014ef-b625-40ad-b5cc-fa72db0c2332 None None] MFA Rules not processed for user `0547f15d00f043b99de10f3db7348c62`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:49.177981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-81c014ef-b625-40ad-b5cc-fa72db0c2332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:49.211146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:49.234102 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1354/2731] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:14:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:49.565327 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21dfa2-8c78-425f-88b4-29b97c3e488d service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:49.566742 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2c21dfa2-8c78-425f-88b4-29b97c3e488d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:49.625178 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21dfa2-8c78-425f-88b4-29b97c3e488d service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:49.629497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f737f04-eefd-4481-a445-fda7c14aa63c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:49.630158 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f737f04-eefd-4481-a445-fda7c14aa63c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:49.630433 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f737f04-eefd-4481-a445-fda7c14aa63c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:49.630700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f737f04-eefd-4481-a445-fda7c14aa63c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:49.632553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5f737f04-eefd-4481-a445-fda7c14aa63c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:49.642760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f737f04-eefd-4481-a445-fda7c14aa63c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:49.644254 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f737f04-eefd-4481-a445-fda7c14aa63c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:49.653507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1378/2732] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:52.741515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c014ef-b625-40ad-b5cc-fa72db0c2332 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:52.743077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-81c014ef-b625-40ad-b5cc-fa72db0c2332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:52.821620 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c014ef-b625-40ad-b5cc-fa72db0c2332 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:52.827354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad7f53-41f1-4de4-9c25-27a266b83fe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:52.827970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad7f53-41f1-4de4-9c25-27a266b83fe6 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:52.828246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad7f53-41f1-4de4-9c25-27a266b83fe6 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:52.828514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad7f53-41f1-4de4-9c25-27a266b83fe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:52.830302 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3dad7f53-41f1-4de4-9c25-27a266b83fe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:52.843145 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad7f53-41f1-4de4-9c25-27a266b83fe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:52.845269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad7f53-41f1-4de4-9c25-27a266b83fe6 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:52.856137 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1355/2733] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:59.649048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=26 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:02.851499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=59 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:11.513043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f737f04-eefd-4481-a445-fda7c14aa63c service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:11.514519 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5f737f04-eefd-4481-a445-fda7c14aa63c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:11.525392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f737f04-eefd-4481-a445-fda7c14aa63c service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:11.529038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888242-e66c-4224-8fe3-5dfc17a97489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:11.529630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888242-e66c-4224-8fe3-5dfc17a97489 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:11.529909 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888242-e66c-4224-8fe3-5dfc17a97489 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:11.530180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888242-e66c-4224-8fe3-5dfc17a97489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:11.532510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-39888242-e66c-4224-8fe3-5dfc17a97489 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:11.596953 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39888242-e66c-4224-8fe3-5dfc17a97489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:11.598430 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39888242-e66c-4224-8fe3-5dfc17a97489 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:11.608434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1379/2734] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:13.498944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad7f53-41f1-4de4-9c25-27a266b83fe6 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:13.510111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad7f53-41f1-4de4-9c25-27a266b83fe6 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:13.516644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304726f-7af7-432c-b5fe-88d9c3c9bd64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:13.517252 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304726f-7af7-432c-b5fe-88d9c3c9bd64 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:13.517529 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304726f-7af7-432c-b5fe-88d9c3c9bd64 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:13.517799 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304726f-7af7-432c-b5fe-88d9c3c9bd64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:13.519583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c304726f-7af7-432c-b5fe-88d9c3c9bd64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:13.534042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304726f-7af7-432c-b5fe-88d9c3c9bd64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:13.535671 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304726f-7af7-432c-b5fe-88d9c3c9bd64 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:13.546969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1356/2735] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:21.603743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=30 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:23.542186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:37.604507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888242-e66c-4224-8fe3-5dfc17a97489 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:37.665668 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888242-e66c-4224-8fe3-5dfc17a97489 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:37.670149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:37.670772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:37.671088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:37.671361 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] PATH_INFO: `/v3/users/fb2dbb409fac421d91e9d7bf0b583ecb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:37.672504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb2dbb409fac421d91e9d7bf0b583ecb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:37.674407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:37.721239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb2dbb409fac421d91e9d7bf0b583ecb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:37.721815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb2dbb409fac421d91e9d7bf0b583ecb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:37.733945 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb2dbb409fac421d91e9d7bf0b583ecb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:37.741103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb2dbb409fac421d91e9d7bf0b583ecb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:37.748411 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb2dbb409fac421d91e9d7bf0b583ecb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:37.754531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1380/2736] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:15:37 2022] DELETE /identity/v3/users/fb2dbb409fac421d91e9d7bf0b583ecb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:37.759239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935366-7587-40ee-8b4b-1b59cad06fe3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:37.759725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935366-7587-40ee-8b4b-1b59cad06fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:37.759923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935366-7587-40ee-8b4b-1b59cad06fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:37.780556 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-31935366-7587-40ee-8b4b-1b59cad06fe3 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:37.782392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31935366-7587-40ee-8b4b-1b59cad06fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:37.842135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1357/2737] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:37.851957 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:37.853464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:37.912301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f868e6d-4db0-433c-9fca-26af6e8797e6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:37.915918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeed6e9-fc39-45c6-b522-ce0d93d58114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:37.916514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed6e9-fc39-45c6-b522-ce0d93d58114 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:37.916791 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed6e9-fc39-45c6-b522-ce0d93d58114 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:37.917059 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed6e9-fc39-45c6-b522-ce0d93d58114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:37.925809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeed6e9-fc39-45c6-b522-ce0d93d58114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:37.927246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeed6e9-fc39-45c6-b522-ce0d93d58114 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:37.936440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1381/2738] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:15:38.208736 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935366-7587-40ee-8b4b-1b59cad06fe3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:38.210186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31935366-7587-40ee-8b4b-1b59cad06fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:38.225279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935366-7587-40ee-8b4b-1b59cad06fe3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:38.229927 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:38.230592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:38.230876 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:38.231233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] PATH_INFO: `/v3/projects/016678f8650943758f2f5f9dc285bf7a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:38.232393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=016678f8650943758f2f5f9dc285bf7a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:38.234280 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:38.284004 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016678f8650943758f2f5f9dc285bf7a', 'request_id': 'req-d20f358b-54e3-4f54-b9fa-66426b117fdc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:38.290440 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016678f8650943758f2f5f9dc285bf7a', 'request_id': 'req-d20f358b-54e3-4f54-b9fa-66426b117fdc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:38.296183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016678f8650943758f2f5f9dc285bf7a', 'request_id': 'req-d20f358b-54e3-4f54-b9fa-66426b117fdc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:38.310091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] The token cache is being invalidate because project 016678f8650943758f2f5f9dc285bf7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:15:38.310382 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:38.311022 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d20f358b-54e3-4f54-b9fa-66426b117fdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:38.313266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1358/2739] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:15:38 2022] DELETE /identity/v3/projects/016678f8650943758f2f5f9dc285bf7a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:38.318127 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b387f5c-31f1-420e-9055-99e3dc72f18b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:38.318483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b387f5c-31f1-420e-9055-99e3dc72f18b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:38.318830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b387f5c-31f1-420e-9055-99e3dc72f18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:38.340862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4b387f5c-31f1-420e-9055-99e3dc72f18b None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:38.342080 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4b387f5c-31f1-420e-9055-99e3dc72f18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:38.361329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:38.409539 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1382/2740] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:38.417914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171a47d-0735-494f-8257-56fa2de7a3b2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:38.418229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171a47d-0735-494f-8257-56fa2de7a3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:38.418477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171a47d-0735-494f-8257-56fa2de7a3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:38.438932 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e171a47d-0735-494f-8257-56fa2de7a3b2 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:38.440164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e171a47d-0735-494f-8257-56fa2de7a3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:38.448503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1359/2741] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:38.453914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b387f5c-31f1-420e-9055-99e3dc72f18b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:38.464544 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b387f5c-31f1-420e-9055-99e3dc72f18b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:38.469360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da247a-d620-40a9-994a-f9f5eceafbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:38.469978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da247a-d620-40a9-994a-f9f5eceafbd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:38.470262 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da247a-d620-40a9-994a-f9f5eceafbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:38.470563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da247a-d620-40a9-994a-f9f5eceafbd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:38.471242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da247a-d620-40a9-994a-f9f5eceafbd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:15:38.471551 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da247a-d620-40a9-994a-f9f5eceafbd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:38.473188 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da247a-d620-40a9-994a-f9f5eceafbd5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:38.489932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1383/2742] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:15:38.491191 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171a47d-0735-494f-8257-56fa2de7a3b2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:38.502119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171a47d-0735-494f-8257-56fa2de7a3b2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:38.507522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4aa20f-04a3-4b7b-9af3-9d00d9e34628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:38.508223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aa20f-04a3-4b7b-9af3-9d00d9e34628 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:38.508685 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aa20f-04a3-4b7b-9af3-9d00d9e34628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:38.508934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aa20f-04a3-4b7b-9af3-9d00d9e34628 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:38.509734 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aa20f-04a3-4b7b-9af3-9d00d9e34628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:38.511869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aa20f-04a3-4b7b-9af3-9d00d9e34628 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:38.539825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1360/2743] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:38 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:38.544037 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da247a-d620-40a9-994a-f9f5eceafbd5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:38.554896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da247a-d620-40a9-994a-f9f5eceafbd5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:38.621325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b4a2a-9b3d-4a5b-97dc-8414f27669aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:38.621973 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b4a2a-9b3d-4a5b-97dc-8414f27669aa admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:38.622228 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b4a2a-9b3d-4a5b-97dc-8414f27669aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:38.622510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b4a2a-9b3d-4a5b-97dc-8414f27669aa admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:38.623161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b4a2a-9b3d-4a5b-97dc-8414f27669aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:38.625105 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b4a2a-9b3d-4a5b-97dc-8414f27669aa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:38.648108 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1384/2744] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:38 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:15:38.652515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4aa20f-04a3-4b7b-9af3-9d00d9e34628 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:38.664149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4aa20f-04a3-4b7b-9af3-9d00d9e34628 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:38.669050 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133f240-9557-4e56-9b5d-01b18acc1217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:38.669716 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133f240-9557-4e56-9b5d-01b18acc1217 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:38.670004 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133f240-9557-4e56-9b5d-01b18acc1217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:38.670277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133f240-9557-4e56-9b5d-01b18acc1217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:38.670926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133f240-9557-4e56-9b5d-01b18acc1217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:38.673108 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133f240-9557-4e56-9b5d-01b18acc1217 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:38.687642 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1361/2745] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:38.692373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b4a2a-9b3d-4a5b-97dc-8414f27669aa admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:38.704919 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b4a2a-9b3d-4a5b-97dc-8414f27669aa admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:38.710724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4469c-1af1-4782-9713-dd91390c0e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:38.711596 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4469c-1af1-4782-9713-dd91390c0e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:38.712009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4469c-1af1-4782-9713-dd91390c0e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:38.712478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4469c-1af1-4782-9713-dd91390c0e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:38.713276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4469c-1af1-4782-9713-dd91390c0e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:38.715141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4469c-1af1-4782-9713-dd91390c0e2a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:38.727633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1385/2746] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:38.732504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133f240-9557-4e56-9b5d-01b18acc1217 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:38.747148 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133f240-9557-4e56-9b5d-01b18acc1217 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:38.752238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ad8af-04c9-4f42-afbd-39eb6072dd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:38.752925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ad8af-04c9-4f42-afbd-39eb6072dd58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:38.753211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ad8af-04c9-4f42-afbd-39eb6072dd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:38.753504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ad8af-04c9-4f42-afbd-39eb6072dd58 admin admin] PATH_INFO: `/v3/projects/9d91f4682cb24894b6b6888a5cbbcd29/users/1ea4332a5a154cd89d0bd7d1f6120fc8/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:38.768981 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ad8af-04c9-4f42-afbd-39eb6072dd58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d91f4682cb24894b6b6888a5cbbcd29, user_id=1ea4332a5a154cd89d0bd7d1f6120fc8, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:38.771848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ad8af-04c9-4f42-afbd-39eb6072dd58 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:38.786089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1362/2747] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:38 2022] PUT /identity/v3/projects/9d91f4682cb24894b6b6888a5cbbcd29/users/1ea4332a5a154cd89d0bd7d1f6120fc8/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:15:38.798959 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7f9ca-7dc4-4793-8424-cf3a124479d4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:38.799337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7f9ca-7dc4-4793-8424-cf3a124479d4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:38.799579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7f9ca-7dc4-4793-8424-cf3a124479d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:38.835641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e7b7f9ca-7dc4-4793-8424-cf3a124479d4 None None] MFA Rules not processed for user `1ea4332a5a154cd89d0bd7d1f6120fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:38.837004 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e7b7f9ca-7dc4-4793-8424-cf3a124479d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:38.905642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1386/2748] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:15:38 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:38.917593 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ad8af-04c9-4f42-afbd-39eb6072dd58 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:38.920325 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a24ad8af-04c9-4f42-afbd-39eb6072dd58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:38.949241 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:38.992238 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ad8af-04c9-4f42-afbd-39eb6072dd58 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:38.996092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda343ec-1a0f-42f4-a64f-1f79bbce1609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:38.996734 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda343ec-1a0f-42f4-a64f-1f79bbce1609 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:38.997014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda343ec-1a0f-42f4-a64f-1f79bbce1609 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:38.997288 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda343ec-1a0f-42f4-a64f-1f79bbce1609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:39.009439 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda343ec-1a0f-42f4-a64f-1f79bbce1609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:39.011036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda343ec-1a0f-42f4-a64f-1f79bbce1609 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:39.022532 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1363/2749] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:38 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:39.758654 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f32b12-765d-42f1-ba13-e06d467a4bbd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:39.759176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f32b12-765d-42f1-ba13-e06d467a4bbd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:39.759275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f32b12-765d-42f1-ba13-e06d467a4bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:39.778680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f1f32b12-765d-42f1-ba13-e06d467a4bbd None None] MFA Rules not processed for user `a982dbba4a324a4f893e427000ce9b93`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:39.779924 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f1f32b12-765d-42f1-ba13-e06d467a4bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:39.792889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:39.838242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1387/2750] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:41.091733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda343ec-1a0f-42f4-a64f-1f79bbce1609 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:41.093306 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fda343ec-1a0f-42f4-a64f-1f79bbce1609 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:41.161819 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda343ec-1a0f-42f4-a64f-1f79bbce1609 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:41.166834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe7d5-9285-4b4a-8880-61627a24fb06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:41.167465 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe7d5-9285-4b4a-8880-61627a24fb06 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:41.167745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe7d5-9285-4b4a-8880-61627a24fb06 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:41.168015 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe7d5-9285-4b4a-8880-61627a24fb06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:41.169724 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-01fbe7d5-9285-4b4a-8880-61627a24fb06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:41.181534 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbe7d5-9285-4b4a-8880-61627a24fb06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:41.183121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbe7d5-9285-4b4a-8880-61627a24fb06 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:41.193769 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1364/2751] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:43.128010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f32b12-765d-42f1-ba13-e06d467a4bbd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:43.129522 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f1f32b12-765d-42f1-ba13-e06d467a4bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:43.190011 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f32b12-765d-42f1-ba13-e06d467a4bbd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:43.194116 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df60b08-9751-4d9b-958b-2e0e5e45f377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:43.194721 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df60b08-9751-4d9b-958b-2e0e5e45f377 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:43.195015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df60b08-9751-4d9b-958b-2e0e5e45f377 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:43.195324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df60b08-9751-4d9b-958b-2e0e5e45f377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:43.197012 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1df60b08-9751-4d9b-958b-2e0e5e45f377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:43.207972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df60b08-9751-4d9b-958b-2e0e5e45f377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:43.209423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df60b08-9751-4d9b-958b-2e0e5e45f377 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:43.217533 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1388/2752] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:50.411663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe7d5-9285-4b4a-8880-61627a24fb06 service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:50.475042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe7d5-9285-4b4a-8880-61627a24fb06 service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:50.479667 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:50.480292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:50.480572 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:50.480848 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] PATH_INFO: `/v3/users/1ea4332a5a154cd89d0bd7d1f6120fc8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:50.481963 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea4332a5a154cd89d0bd7d1f6120fc8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:50.483907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:50.530002 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea4332a5a154cd89d0bd7d1f6120fc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:50.535835 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:50.543793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea4332a5a154cd89d0bd7d1f6120fc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:50.544500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea4332a5a154cd89d0bd7d1f6120fc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:50.545138 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea4332a5a154cd89d0bd7d1f6120fc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:50.551704 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea4332a5a154cd89d0bd7d1f6120fc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:50.564279 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1365/2753] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:15:50 2022] DELETE /identity/v3/users/1ea4332a5a154cd89d0bd7d1f6120fc8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:50.568732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d485bd-4f49-446c-9c80-c3caa9f09344 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:50.569213 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d485bd-4f49-446c-9c80-c3caa9f09344 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:50.569314 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d485bd-4f49-446c-9c80-c3caa9f09344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:50.588094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c5d485bd-4f49-446c-9c80-c3caa9f09344 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:50.589275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c5d485bd-4f49-446c-9c80-c3caa9f09344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:50.647383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1389/2754] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:15:50.657441 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:50.658859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:50.719906 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59872a8-12a3-4f5e-a8d5-7cecc07174fb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:50.723569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbe31d-68ee-47b3-90a8-e43f83f24f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:50.724175 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbe31d-68ee-47b3-90a8-e43f83f24f9c service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:50.724543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbe31d-68ee-47b3-90a8-e43f83f24f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:50.724822 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbe31d-68ee-47b3-90a8-e43f83f24f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:50.738336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbe31d-68ee-47b3-90a8-e43f83f24f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:50.739908 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbe31d-68ee-47b3-90a8-e43f83f24f9c service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:50.750910 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1366/2755] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:51.052079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d485bd-4f49-446c-9c80-c3caa9f09344 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:51.053739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c5d485bd-4f49-446c-9c80-c3caa9f09344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:51.066090 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d485bd-4f49-446c-9c80-c3caa9f09344 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:51.070726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:51.071415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.071698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.071969 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] PATH_INFO: `/v3/projects/9d91f4682cb24894b6b6888a5cbbcd29` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.073220 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d91f4682cb24894b6b6888a5cbbcd29)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:51.075157 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:51.122369 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d91f4682cb24894b6b6888a5cbbcd29', 'request_id': 'req-17dc3d18-c953-4c15-82f0-25ce242a4bbf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:51.129937 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d91f4682cb24894b6b6888a5cbbcd29', 'request_id': 'req-17dc3d18-c953-4c15-82f0-25ce242a4bbf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:51.136405 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d91f4682cb24894b6b6888a5cbbcd29', 'request_id': 'req-17dc3d18-c953-4c15-82f0-25ce242a4bbf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:51.137592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:51.149109 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] The token cache is being invalidate because project 9d91f4682cb24894b6b6888a5cbbcd29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:15:51.149483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:51.150149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-17dc3d18-c953-4c15-82f0-25ce242a4bbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:51.151794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1390/2756] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:15:51 2022] DELETE /identity/v3/projects/9d91f4682cb24894b6b6888a5cbbcd29 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:51.157278 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dcb43-4ff1-4c1d-b819-8675656b0c25 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.157562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dcb43-4ff1-4c1d-b819-8675656b0c25 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.157841 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dcb43-4ff1-4c1d-b819-8675656b0c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.177266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6a3dcb43-4ff1-4c1d-b819-8675656b0c25 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:51.178457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6a3dcb43-4ff1-4c1d-b819-8675656b0c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:51.236732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1367/2757] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:51.244581 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d518e-dbf4-424c-a595-737af4f84c3a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.244886 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d518e-dbf4-424c-a595-737af4f84c3a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.245164 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d518e-dbf4-424c-a595-737af4f84c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.265267 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b89d518e-dbf4-424c-a595-737af4f84c3a None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:51.266437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b89d518e-dbf4-424c-a595-737af4f84c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:51.274203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1391/2758] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:51.278358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dcb43-4ff1-4c1d-b819-8675656b0c25 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:51.287927 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dcb43-4ff1-4c1d-b819-8675656b0c25 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:51.292495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd2718-57b7-4036-9622-9cb8e5f94c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:51.293078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd2718-57b7-4036-9622-9cb8e5f94c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.293358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd2718-57b7-4036-9622-9cb8e5f94c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.293632 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd2718-57b7-4036-9622-9cb8e5f94c8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.294270 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd2718-57b7-4036-9622-9cb8e5f94c8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:15:51.294570 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd2718-57b7-4036-9622-9cb8e5f94c8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:51.296169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd2718-57b7-4036-9622-9cb8e5f94c8a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:51.307772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1368/2759] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:15:51.311990 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d518e-dbf4-424c-a595-737af4f84c3a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:51.321819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d518e-dbf4-424c-a595-737af4f84c3a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:51.326613 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3d3f5-0a00-4ef0-a06b-057c79350ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:51.327523 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3d3f5-0a00-4ef0-a06b-057c79350ced admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.327803 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3d3f5-0a00-4ef0-a06b-057c79350ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.328251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3d3f5-0a00-4ef0-a06b-057c79350ced admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.329249 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3d3f5-0a00-4ef0-a06b-057c79350ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:51.331321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3d3f5-0a00-4ef0-a06b-057c79350ced admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:51.358787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1392/2760] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:51 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:15:51.363922 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd2718-57b7-4036-9622-9cb8e5f94c8a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:51.373866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd2718-57b7-4036-9622-9cb8e5f94c8a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:51.429546 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b1952-cd39-4a5b-ad4e-c3a4048a9886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:51.430180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b1952-cd39-4a5b-ad4e-c3a4048a9886 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.430392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b1952-cd39-4a5b-ad4e-c3a4048a9886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.430646 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b1952-cd39-4a5b-ad4e-c3a4048a9886 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.431411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b1952-cd39-4a5b-ad4e-c3a4048a9886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:51.433314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b1952-cd39-4a5b-ad4e-c3a4048a9886 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:51.442172 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:51.456151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1369/2761] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:51 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:51.460459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3d3f5-0a00-4ef0-a06b-057c79350ced admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:51.471770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3d3f5-0a00-4ef0-a06b-057c79350ced admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:51.477257 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904edc57-0ce5-4ca2-a18b-89a2d71b645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:51.477868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904edc57-0ce5-4ca2-a18b-89a2d71b645f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.478152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904edc57-0ce5-4ca2-a18b-89a2d71b645f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.478541 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904edc57-0ce5-4ca2-a18b-89a2d71b645f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.479234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904edc57-0ce5-4ca2-a18b-89a2d71b645f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:51.480823 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904edc57-0ce5-4ca2-a18b-89a2d71b645f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:51.491718 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1393/2762] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:51.495652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b1952-cd39-4a5b-ad4e-c3a4048a9886 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:51.505618 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b1952-cd39-4a5b-ad4e-c3a4048a9886 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:51.510448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a1bc8-2122-4773-83f3-2f0400f7cc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:51.511061 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a1bc8-2122-4773-83f3-2f0400f7cc08 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.511341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a1bc8-2122-4773-83f3-2f0400f7cc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.511617 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a1bc8-2122-4773-83f3-2f0400f7cc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.512250 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a1bc8-2122-4773-83f3-2f0400f7cc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:51.513816 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a1bc8-2122-4773-83f3-2f0400f7cc08 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:51.525331 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1370/2763] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:51.529535 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904edc57-0ce5-4ca2-a18b-89a2d71b645f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:51.539994 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904edc57-0ce5-4ca2-a18b-89a2d71b645f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:51.544664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70e1df-12a1-4d52-a12e-6c1429310d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:51.545306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70e1df-12a1-4d52-a12e-6c1429310d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.545587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70e1df-12a1-4d52-a12e-6c1429310d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.545862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70e1df-12a1-4d52-a12e-6c1429310d06 admin admin] PATH_INFO: `/v3/projects/bd38972fd2dc4dda987f96f0a6aedcc7/users/a4fe2e98ec3d42ccadd648a53fd7b074/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.561618 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70e1df-12a1-4d52-a12e-6c1429310d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd38972fd2dc4dda987f96f0a6aedcc7, user_id=a4fe2e98ec3d42ccadd648a53fd7b074, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:51.564331 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70e1df-12a1-4d52-a12e-6c1429310d06 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:51.578243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1394/2764] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:51 2022] PUT /identity/v3/projects/bd38972fd2dc4dda987f96f0a6aedcc7/users/a4fe2e98ec3d42ccadd648a53fd7b074/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:51.583236 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd369a4-186e-4ad3-9b14-2c1fc6d4c8d6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.583523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd369a4-186e-4ad3-9b14-2c1fc6d4c8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.583792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd369a4-186e-4ad3-9b14-2c1fc6d4c8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.602724 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-afd369a4-186e-4ad3-9b14-2c1fc6d4c8d6 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:51.603983 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-afd369a4-186e-4ad3-9b14-2c1fc6d4c8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:51.661943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1371/2765] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:51.672296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70e1df-12a1-4d52-a12e-6c1429310d06 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:51.673992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3e70e1df-12a1-4d52-a12e-6c1429310d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:51.733530 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70e1df-12a1-4d52-a12e-6c1429310d06 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:51.737408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da94e1-9e0a-41b1-b41b-8efe8a8e98e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:51.738018 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da94e1-9e0a-41b1-b41b-8efe8a8e98e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.738298 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da94e1-9e0a-41b1-b41b-8efe8a8e98e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.738569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da94e1-9e0a-41b1-b41b-8efe8a8e98e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.747604 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da94e1-9e0a-41b1-b41b-8efe8a8e98e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:51.749029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da94e1-9e0a-41b1-b41b-8efe8a8e98e6 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:51.758301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1395/2766] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:56.765213 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e07e-8d06-42d9-930b-566f565cda9f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:56.766346 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e07e-8d06-42d9-930b-566f565cda9f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:56.766555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e07e-8d06-42d9-930b-566f565cda9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:56.796691 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c8c9e07e-8d06-42d9-930b-566f565cda9f None None] MFA Rules not processed for user `a4fe2e98ec3d42ccadd648a53fd7b074`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:56.797916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c8c9e07e-8d06-42d9-930b-566f565cda9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:56.883951 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1372/2767] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:15:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:56.888415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da94e1-9e0a-41b1-b41b-8efe8a8e98e6 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:56.906650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da94e1-9e0a-41b1-b41b-8efe8a8e98e6 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:56.911355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f905-5138-415d-bc9f-3804dcc2c89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:56.911961 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f905-5138-415d-bc9f-3804dcc2c89c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:56.912239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f905-5138-415d-bc9f-3804dcc2c89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:56.912511 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f905-5138-415d-bc9f-3804dcc2c89c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:56.913152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02f905-5138-415d-bc9f-3804dcc2c89c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:56.915102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02f905-5138-415d-bc9f-3804dcc2c89c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:56.947305 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1396/2768] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:56 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:56.948682 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9e07e-8d06-42d9-930b-566f565cda9f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:56.960082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9e07e-8d06-42d9-930b-566f565cda9f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:57.021023 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796e717-49b0-4b78-9363-76656be19c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:57.021639 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796e717-49b0-4b78-9363-76656be19c0c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:57.021919 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796e717-49b0-4b78-9363-76656be19c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:57.022194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796e717-49b0-4b78-9363-76656be19c0c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:57.022825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796e717-49b0-4b78-9363-76656be19c0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:57.024763 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796e717-49b0-4b78-9363-76656be19c0c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:57.044718 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:57.055104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1373/2769] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:56 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:57.056509 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f905-5138-415d-bc9f-3804dcc2c89c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:57.068346 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f905-5138-415d-bc9f-3804dcc2c89c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:57.073246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64de38c-69db-4725-b96c-c36f92b874b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:57.073872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64de38c-69db-4725-b96c-c36f92b874b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:57.074155 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64de38c-69db-4725-b96c-c36f92b874b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:57.074428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64de38c-69db-4725-b96c-c36f92b874b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:57.075117 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64de38c-69db-4725-b96c-c36f92b874b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:57.076736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64de38c-69db-4725-b96c-c36f92b874b3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:57.088936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1397/2770] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:57.093269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796e717-49b0-4b78-9363-76656be19c0c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:57.104434 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796e717-49b0-4b78-9363-76656be19c0c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:57.110019 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b817274-f0e1-411d-887c-53b5286e63f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:57.110717 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b817274-f0e1-411d-887c-53b5286e63f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:57.110980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b817274-f0e1-411d-887c-53b5286e63f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:57.111281 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b817274-f0e1-411d-887c-53b5286e63f5 admin admin] PATH_INFO: `/v3/projects/4055ec1f68d5433db9f16810d68b992b/users/e30fac9da61c4d06b0008e445ad7eaa9/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:57.129710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b817274-f0e1-411d-887c-53b5286e63f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4055ec1f68d5433db9f16810d68b992b, user_id=e30fac9da61c4d06b0008e445ad7eaa9, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:57.132468 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b817274-f0e1-411d-887c-53b5286e63f5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:57.148262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1374/2771] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:57 2022] PUT /identity/v3/projects/4055ec1f68d5433db9f16810d68b992b/users/e30fac9da61c4d06b0008e445ad7eaa9/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:15:57.152189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64de38c-69db-4725-b96c-c36f92b874b3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:57.162667 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64de38c-69db-4725-b96c-c36f92b874b3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:57.218502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:57.226931 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449214a7-f7d8-4728-a28e-7c20acda4fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:57.227851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449214a7-f7d8-4728-a28e-7c20acda4fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:57.228146 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449214a7-f7d8-4728-a28e-7c20acda4fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:57.228757 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449214a7-f7d8-4728-a28e-7c20acda4fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:57.229601 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449214a7-f7d8-4728-a28e-7c20acda4fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:57.231432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449214a7-f7d8-4728-a28e-7c20acda4fc3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:57.244391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1398/2772] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:57.248837 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b817274-f0e1-411d-887c-53b5286e63f5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:57.260475 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b817274-f0e1-411d-887c-53b5286e63f5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:57.265515 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442b909-fded-44a5-abb1-e4409e23a4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:57.266207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442b909-fded-44a5-abb1-e4409e23a4cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:57.266486 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442b909-fded-44a5-abb1-e4409e23a4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:57.266760 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442b909-fded-44a5-abb1-e4409e23a4cb admin admin] PATH_INFO: `/v3/projects/4055ec1f68d5433db9f16810d68b992b/users/e30fac9da61c4d06b0008e445ad7eaa9/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:57.269417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442b909-fded-44a5-abb1-e4409e23a4cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4055ec1f68d5433db9f16810d68b992b, user_id=e30fac9da61c4d06b0008e445ad7eaa9, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:57.272091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442b909-fded-44a5-abb1-e4409e23a4cb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:57.283522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1375/2773] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:57 2022] PUT /identity/v3/projects/4055ec1f68d5433db9f16810d68b992b/users/e30fac9da61c4d06b0008e445ad7eaa9/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:02.829788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d85218-7444-4df5-985c-3735914d2e92 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:02.830328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d85218-7444-4df5-985c-3735914d2e92 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:02.830328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d85218-7444-4df5-985c-3735914d2e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:02.858375 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e5d85218-7444-4df5-985c-3735914d2e92 None None] MFA Rules not processed for user `e30fac9da61c4d06b0008e445ad7eaa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:02.860599 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e5d85218-7444-4df5-985c-3735914d2e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:02.929500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1399/2774] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:16:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:02.942312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442b909-fded-44a5-abb1-e4409e23a4cb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:02.944584 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d442b909-fded-44a5-abb1-e4409e23a4cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:03.036251 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442b909-fded-44a5-abb1-e4409e23a4cb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:03.039880 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11986142-8c65-4359-b788-a920e987f580 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af69888b04c45b09e34d9ff7a0b564f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:03.040477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11986142-8c65-4359-b788-a920e987f580 service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:03.040751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11986142-8c65-4359-b788-a920e987f580 service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:03.041019 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11986142-8c65-4359-b788-a920e987f580 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:03.097482 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11986142-8c65-4359-b788-a920e987f580 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:03.098870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11986142-8c65-4359-b788-a920e987f580 service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:03.107825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1376/2775] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:03.236334 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb5977-a7be-40ef-b06e-96e4031a1f4e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:03.236902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb5977-a7be-40ef-b06e-96e4031a1f4e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:03.237272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb5977-a7be-40ef-b06e-96e4031a1f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:03.257602 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-efdb5977-a7be-40ef-b06e-96e4031a1f4e None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:03.258949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-efdb5977-a7be-40ef-b06e-96e4031a1f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:03.352143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1400/2776] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:16:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:03.427035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11986142-8c65-4359-b788-a920e987f580 service glance] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:03.428615 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-11986142-8c65-4359-b788-a920e987f580 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:03.539268 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:03.567184 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11986142-8c65-4359-b788-a920e987f580 service glance] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:03.572298 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c910c3-dc45-4b94-b2d1-064b6071dd10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:03.572943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c910c3-dc45-4b94-b2d1-064b6071dd10 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:03.573220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c910c3-dc45-4b94-b2d1-064b6071dd10 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:03.573491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c910c3-dc45-4b94-b2d1-064b6071dd10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:03.583261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c910c3-dc45-4b94-b2d1-064b6071dd10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:03.584859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c910c3-dc45-4b94-b2d1-064b6071dd10 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:03.593674 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1377/2777] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:16:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:16:12.513160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67f7cb-b5dc-45d2-9122-3b08c389b3cc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:12.513687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67f7cb-b5dc-45d2-9122-3b08c389b3cc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:12.513687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67f7cb-b5dc-45d2-9122-3b08c389b3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:12.532987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fe67f7cb-b5dc-45d2-9122-3b08c389b3cc None None] MFA Rules not processed for user `a4fe2e98ec3d42ccadd648a53fd7b074`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:12.534181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fe67f7cb-b5dc-45d2-9122-3b08c389b3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:12.541123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1401/2778] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:16:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:13.590162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=11 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:14.685766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ce7ce-81f0-4783-a8ba-892605ff9615 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:14.686327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ce7ce-81f0-4783-a8ba-892605ff9615 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:14.686327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ce7ce-81f0-4783-a8ba-892605ff9615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:14.706548 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-da9ce7ce-81f0-4783-a8ba-892605ff9615 None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:14.707821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-da9ce7ce-81f0-4783-a8ba-892605ff9615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:14.715510 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1378/2779] 10.222.0.86 () {58 vars in 1009 bytes} [Mon Aug 29 19:16:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:14.728331 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67f7cb-b5dc-45d2-9122-3b08c389b3cc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:14.741854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67f7cb-b5dc-45d2-9122-3b08c389b3cc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:14.746982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e9c65-a07c-41dd-89b3-53e4d1560f97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:14.747614 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e9c65-a07c-41dd-89b3-53e4d1560f97 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:14.747942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e9c65-a07c-41dd-89b3-53e4d1560f97 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:14.748296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e9c65-a07c-41dd-89b3-53e4d1560f97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:14.757752 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e9c65-a07c-41dd-89b3-53e4d1560f97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:14.759429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e9c65-a07c-41dd-89b3-53e4d1560f97 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:14.768182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1402/2780] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:16:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:16:24.701583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:24.765308 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=72 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:39.065273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ce7ce-81f0-4783-a8ba-892605ff9615 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:39.120226 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ce7ce-81f0-4783-a8ba-892605ff9615 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:39.123846 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc0106-19c6-4af2-aead-3683b24964a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:39.124448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc0106-19c6-4af2-aead-3683b24964a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:39.124727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc0106-19c6-4af2-aead-3683b24964a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:39.124996 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc0106-19c6-4af2-aead-3683b24964a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:39.127372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8dbc0106-19c6-4af2-aead-3683b24964a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:39.196663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc0106-19c6-4af2-aead-3683b24964a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:39.198079 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc0106-19c6-4af2-aead-3683b24964a3 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:39.209544 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1379/2781] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:40.391854 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e9c65-a07c-41dd-89b3-53e4d1560f97 service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:40.401568 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e9c65-a07c-41dd-89b3-53e4d1560f97 service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:40.405202 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff038c-00c2-4a4f-be60-88cb46835681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:40.405787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff038c-00c2-4a4f-be60-88cb46835681 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:40.406062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff038c-00c2-4a4f-be60-88cb46835681 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:40.406332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff038c-00c2-4a4f-be60-88cb46835681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:40.408057 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c6ff038c-00c2-4a4f-be60-88cb46835681 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:40.421763 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff038c-00c2-4a4f-be60-88cb46835681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:40.424118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff038c-00c2-4a4f-be60-88cb46835681 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:40.435815 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1403/2782] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:49.204425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=54 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:50.430661 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:53.080514 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc0106-19c6-4af2-aead-3683b24964a3 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:53.089403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc0106-19c6-4af2-aead-3683b24964a3 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:53.093003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e998c-85a7-4f53-a0e7-38c460f40431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:53.093598 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e998c-85a7-4f53-a0e7-38c460f40431 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:53.093874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e998c-85a7-4f53-a0e7-38c460f40431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:53.094170 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e998c-85a7-4f53-a0e7-38c460f40431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:53.103111 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e998c-85a7-4f53-a0e7-38c460f40431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:53.104508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e998c-85a7-4f53-a0e7-38c460f40431 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:53.113817 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1380/2783] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:17:03.109587 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:17:46.225313 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff038c-00c2-4a4f-be60-88cb46835681 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:17:46.235783 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff038c-00c2-4a4f-be60-88cb46835681 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:17:46.240037 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c5e3e-c02e-4586-8b1d-7103a19b0dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:17:46.241408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c5e3e-c02e-4586-8b1d-7103a19b0dee service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:17:46.241697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c5e3e-c02e-4586-8b1d-7103a19b0dee service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:17:46.242154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c5e3e-c02e-4586-8b1d-7103a19b0dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:17:46.244381 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f95c5e3e-c02e-4586-8b1d-7103a19b0dee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:17:46.258110 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c5e3e-c02e-4586-8b1d-7103a19b0dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:17:46.259808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c5e3e-c02e-4586-8b1d-7103a19b0dee service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:17:46.270569 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1404/2784] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:17:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:17:56.266297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:01.339960 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53b9d5-9814-4758-ac2f-0eded323ea91 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:01.340520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53b9d5-9814-4758-ac2f-0eded323ea91 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:01.340520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53b9d5-9814-4758-ac2f-0eded323ea91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:01.359248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7d53b9d5-9814-4758-ac2f-0eded323ea91 None None] MFA Rules not processed for user `a982dbba4a324a4f893e427000ce9b93`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:18:01.360437 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7d53b9d5-9814-4758-ac2f-0eded323ea91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:01.413894 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1381/2785] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:18:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:18:01.682965 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c5e3e-c02e-4586-8b1d-7103a19b0dee service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:01.685063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f95c5e3e-c02e-4586-8b1d-7103a19b0dee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:01.697762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c5e3e-c02e-4586-8b1d-7103a19b0dee service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:01.701632 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ad5dd-6fa1-4e41-aaf2-ec4ee29cc27c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:01.702243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ad5dd-6fa1-4e41-aaf2-ec4ee29cc27c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:01.702526 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ad5dd-6fa1-4e41-aaf2-ec4ee29cc27c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:01.702797 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ad5dd-6fa1-4e41-aaf2-ec4ee29cc27c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:01.704862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e65ad5dd-6fa1-4e41-aaf2-ec4ee29cc27c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:01.770568 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ad5dd-6fa1-4e41-aaf2-ec4ee29cc27c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:01.772079 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ad5dd-6fa1-4e41-aaf2-ec4ee29cc27c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:01.782103 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1405/2786] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:03.059834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53b9d5-9814-4758-ac2f-0eded323ea91 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:03.062394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7d53b9d5-9814-4758-ac2f-0eded323ea91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:03.138001 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53b9d5-9814-4758-ac2f-0eded323ea91 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:03.144016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177af518-c536-4a8e-a916-9620e7cf2474 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:03.144565 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177af518-c536-4a8e-a916-9620e7cf2474 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:03.145055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177af518-c536-4a8e-a916-9620e7cf2474 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:03.145192 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177af518-c536-4a8e-a916-9620e7cf2474 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:03.146909 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-177af518-c536-4a8e-a916-9620e7cf2474 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:03.160954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177af518-c536-4a8e-a916-9620e7cf2474 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:03.163087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177af518-c536-4a8e-a916-9620e7cf2474 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:03.175159 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1382/2787] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:04.939926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ad5dd-6fa1-4e41-aaf2-ec4ee29cc27c service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:04.941422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e65ad5dd-6fa1-4e41-aaf2-ec4ee29cc27c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:04.957263 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ad5dd-6fa1-4e41-aaf2-ec4ee29cc27c service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:04.963232 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb85961-a518-44f8-aa91-a31af7f72029 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:04.963904 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb85961-a518-44f8-aa91-a31af7f72029 service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:04.964190 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb85961-a518-44f8-aa91-a31af7f72029 service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:04.964515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb85961-a518-44f8-aa91-a31af7f72029 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:04.966373 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eeb85961-a518-44f8-aa91-a31af7f72029 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:04.979577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb85961-a518-44f8-aa91-a31af7f72029 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:04.982610 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb85961-a518-44f8-aa91-a31af7f72029 service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:04.996766 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1406/2788] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:06.543477 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177af518-c536-4a8e-a916-9620e7cf2474 service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:06.555359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177af518-c536-4a8e-a916-9620e7cf2474 service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:06.560058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f254258-0873-41ba-9846-fd8000324361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:06.560965 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f254258-0873-41ba-9846-fd8000324361 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:06.561402 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f254258-0873-41ba-9846-fd8000324361 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:06.561827 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f254258-0873-41ba-9846-fd8000324361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:06.563907 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5f254258-0873-41ba-9846-fd8000324361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:06.657768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f254258-0873-41ba-9846-fd8000324361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:06.659555 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f254258-0873-41ba-9846-fd8000324361 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:06.702850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1383/2789] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:08.486020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb85961-a518-44f8-aa91-a31af7f72029 service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:08.487510 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eeb85961-a518-44f8-aa91-a31af7f72029 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:08.510908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb85961-a518-44f8-aa91-a31af7f72029 service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:08.516788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:08.519671 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:08.519880 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:08.526787 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] PATH_INFO: `/v3/users/0547f15d00f043b99de10f3db7348c62` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:08.529504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0547f15d00f043b99de10f3db7348c62)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:08.531495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:08.614741 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0547f15d00f043b99de10f3db7348c62'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:08.616328 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0547f15d00f043b99de10f3db7348c62'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:08.635015 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0547f15d00f043b99de10f3db7348c62'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:08.649406 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0547f15d00f043b99de10f3db7348c62'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:08.667208 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0547f15d00f043b99de10f3db7348c62'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:08.683905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1407/2790] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:18:08 2022] DELETE /identity/v3/users/0547f15d00f043b99de10f3db7348c62 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:18:09.036425 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f254258-0873-41ba-9846-fd8000324361 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:09.066277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f254258-0873-41ba-9846-fd8000324361 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:09.158627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:09.189751 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:09.190405 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:09.190679 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:09.190954 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] PATH_INFO: `/v3/projects/5d70fdff5c5a443b9f0ae1a24dfffcd3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:09.192112 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d70fdff5c5a443b9f0ae1a24dfffcd3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:09.194284 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:09.245891 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d70fdff5c5a443b9f0ae1a24dfffcd3', 'request_id': 'req-b6f66854-deac-4c3c-b05b-40ae653b4886'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.252795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d70fdff5c5a443b9f0ae1a24dfffcd3', 'request_id': 'req-b6f66854-deac-4c3c-b05b-40ae653b4886'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.258821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d70fdff5c5a443b9f0ae1a24dfffcd3', 'request_id': 'req-b6f66854-deac-4c3c-b05b-40ae653b4886'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.272627 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] The token cache is being invalidate because project 5d70fdff5c5a443b9f0ae1a24dfffcd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:18:09.273055 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.274201 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.276219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1384/2791] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:18:09 2022] DELETE /identity/v3/projects/5d70fdff5c5a443b9f0ae1a24dfffcd3 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:18:09.280004 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:09.281437 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:09.298148 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c941de6-c95e-477a-86a2-88183808ece5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:09.360045 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:09.372364 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:09.373296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:09.374312 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:09.374851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] PATH_INFO: `/v3/users/81d3c2ac625a4cffbdde0da4be64a65e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:09.376694 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d3c2ac625a4cffbdde0da4be64a65e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:09.380698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:09.433054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d3c2ac625a4cffbdde0da4be64a65e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.433736 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d3c2ac625a4cffbdde0da4be64a65e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.446215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d3c2ac625a4cffbdde0da4be64a65e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.455351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d3c2ac625a4cffbdde0da4be64a65e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.463119 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d3c2ac625a4cffbdde0da4be64a65e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.469185 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1408/2792] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:18:09 2022] DELETE /identity/v3/users/81d3c2ac625a4cffbdde0da4be64a65e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:18:09.648336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:09.649925 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:09.665006 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f66854-deac-4c3c-b05b-40ae653b4886 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:09.730305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:09.731078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:09.731427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:09.731771 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] PATH_INFO: `/v3/projects/91b7f611c9ad43d4ac9f13506c654f76` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:09.733045 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91b7f611c9ad43d4ac9f13506c654f76)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:09.735059 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:09.790105 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91b7f611c9ad43d4ac9f13506c654f76', 'request_id': 'req-fed8df0e-d102-416a-ba7b-dbc92c9e6963'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.796487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91b7f611c9ad43d4ac9f13506c654f76', 'request_id': 'req-fed8df0e-d102-416a-ba7b-dbc92c9e6963'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.802668 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91b7f611c9ad43d4ac9f13506c654f76', 'request_id': 'req-fed8df0e-d102-416a-ba7b-dbc92c9e6963'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.805547 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:09.818713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] The token cache is being invalidate because project 91b7f611c9ad43d4ac9f13506c654f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:18:09.819020 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.819683 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:09.821179 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1385/2793] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:18:09 2022] DELETE /identity/v3/projects/91b7f611c9ad43d4ac9f13506c654f76 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:18:15.173342 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:15.176655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:15.236415 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0484d5-3fa9-42cd-907b-3c061ecf744f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:15.284813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbf484-9b87-4438-b49e-6dbd8075ee0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:15.285338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbf484-9b87-4438-b49e-6dbd8075ee0c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:15.286102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbf484-9b87-4438-b49e-6dbd8075ee0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:15.286392 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbf484-9b87-4438-b49e-6dbd8075ee0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:15.288118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d7dbf484-9b87-4438-b49e-6dbd8075ee0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:15.330731 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:15.352819 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbf484-9b87-4438-b49e-6dbd8075ee0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:15.354244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbf484-9b87-4438-b49e-6dbd8075ee0c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:15.364183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1409/2794] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:16.240341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:16.242215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:16.345412 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8df0e-d102-416a-ba7b-dbc92c9e6963 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:16.352432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0952b1-e0db-4651-aa2e-b5347dac7cc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:16.353304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0952b1-e0db-4651-aa2e-b5347dac7cc3 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:16.353685 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0952b1-e0db-4651-aa2e-b5347dac7cc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:16.354053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0952b1-e0db-4651-aa2e-b5347dac7cc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:16.356870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9b0952b1-e0db-4651-aa2e-b5347dac7cc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:16.378245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0952b1-e0db-4651-aa2e-b5347dac7cc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:16.380387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0952b1-e0db-4651-aa2e-b5347dac7cc3 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:16.393670 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1386/2795] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:16.503215 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbf484-9b87-4438-b49e-6dbd8075ee0c service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:16.520706 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbf484-9b87-4438-b49e-6dbd8075ee0c service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:16.533467 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1734796-cd8b-4b69-84f4-58e62dffb459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:16.534095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734796-cd8b-4b69-84f4-58e62dffb459 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:16.534570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734796-cd8b-4b69-84f4-58e62dffb459 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:16.535074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734796-cd8b-4b69-84f4-58e62dffb459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:16.537504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c1734796-cd8b-4b69-84f4-58e62dffb459 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:16.619160 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1734796-cd8b-4b69-84f4-58e62dffb459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:16.621168 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1734796-cd8b-4b69-84f4-58e62dffb459 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:16.632269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1410/2796] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:22.949401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0952b1-e0db-4651-aa2e-b5347dac7cc3 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:22.950851 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9b0952b1-e0db-4651-aa2e-b5347dac7cc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:22.965481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0952b1-e0db-4651-aa2e-b5347dac7cc3 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:22.970631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:22.971359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:22.971765 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:22.972047 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] PATH_INFO: `/v3/users/260ffd33f3c443c09fdb8d27a1de846e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:22.973175 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=260ffd33f3c443c09fdb8d27a1de846e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:22.975134 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:23.022350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '260ffd33f3c443c09fdb8d27a1de846e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.034211 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '260ffd33f3c443c09fdb8d27a1de846e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.034881 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '260ffd33f3c443c09fdb8d27a1de846e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.035617 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '260ffd33f3c443c09fdb8d27a1de846e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.043017 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '260ffd33f3c443c09fdb8d27a1de846e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.055994 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1387/2797] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:18:22 2022] DELETE /identity/v3/users/260ffd33f3c443c09fdb8d27a1de846e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:18:23.233234 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1734796-cd8b-4b69-84f4-58e62dffb459 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:23.245892 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1734796-cd8b-4b69-84f4-58e62dffb459 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:23.303217 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:23.303762 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:23.304039 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:23.304308 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] PATH_INFO: `/v3/projects/3d7f020293da4a45a541d6ca1397380d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:23.305408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d7f020293da4a45a541d6ca1397380d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:23.307322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:23.354014 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d7f020293da4a45a541d6ca1397380d', 'request_id': 'req-93998e6f-eb0f-4114-aef8-cad495f29e85'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.361258 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d7f020293da4a45a541d6ca1397380d', 'request_id': 'req-93998e6f-eb0f-4114-aef8-cad495f29e85'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.369090 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d7f020293da4a45a541d6ca1397380d', 'request_id': 'req-93998e6f-eb0f-4114-aef8-cad495f29e85'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.379445 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] The token cache is being invalidate because project 3d7f020293da4a45a541d6ca1397380d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:18:23.379735 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.380377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.381852 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1411/2798] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:18:23 2022] DELETE /identity/v3/projects/3d7f020293da4a45a541d6ca1397380d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:18:23.385864 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:23.387336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:23.400370 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa721b-fdde-4997-81a6-f07dc20f7302 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:23.446759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:23.457713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39044940-84c6-42ac-8487-c0d0949fa9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:23.458366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39044940-84c6-42ac-8487-c0d0949fa9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:23.458641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39044940-84c6-42ac-8487-c0d0949fa9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:23.458909 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39044940-84c6-42ac-8487-c0d0949fa9a4 admin admin] PATH_INFO: `/v3/users/a96c7820b4724e8eb0becb5d736445a9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:23.460053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39044940-84c6-42ac-8487-c0d0949fa9a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a96c7820b4724e8eb0becb5d736445a9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:23.461945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39044940-84c6-42ac-8487-c0d0949fa9a4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:23.514007 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-39044940-84c6-42ac-8487-c0d0949fa9a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a96c7820b4724e8eb0becb5d736445a9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.526975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-39044940-84c6-42ac-8487-c0d0949fa9a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a96c7820b4724e8eb0becb5d736445a9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.528024 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-39044940-84c6-42ac-8487-c0d0949fa9a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a96c7820b4724e8eb0becb5d736445a9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.528486 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-39044940-84c6-42ac-8487-c0d0949fa9a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a96c7820b4724e8eb0becb5d736445a9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.535511 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-39044940-84c6-42ac-8487-c0d0949fa9a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a96c7820b4724e8eb0becb5d736445a9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.550392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1388/2799] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:18:23 2022] DELETE /identity/v3/users/a96c7820b4724e8eb0becb5d736445a9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:18:23.723395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:23.725579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:23.739091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93998e6f-eb0f-4114-aef8-cad495f29e85 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:23.742932 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:23.797700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:23.798332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:23.798613 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:23.798885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] PATH_INFO: `/v3/projects/94fcc042687d4af0b8d8428c56bb60ef` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:23.800027 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94fcc042687d4af0b8d8428c56bb60ef)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:23.802434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:23.852062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94fcc042687d4af0b8d8428c56bb60ef', 'request_id': 'req-0085d319-1ae9-4e17-899f-bceeb0726572'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.860229 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94fcc042687d4af0b8d8428c56bb60ef', 'request_id': 'req-0085d319-1ae9-4e17-899f-bceeb0726572'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.867474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94fcc042687d4af0b8d8428c56bb60ef', 'request_id': 'req-0085d319-1ae9-4e17-899f-bceeb0726572'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.886344 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] The token cache is being invalidate because project 94fcc042687d4af0b8d8428c56bb60ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:18:23.886703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.887452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:23.889096 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1412/2800] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:18:23 2022] DELETE /identity/v3/projects/94fcc042687d4af0b8d8428c56bb60ef => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:18:33.546082 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:33.874862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:19:39.360464 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec72d-fdf6-4153-a209-f28398f5380c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:39.361002 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec72d-fdf6-4153-a209-f28398f5380c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:39.361319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec72d-fdf6-4153-a209-f28398f5380c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:39.397444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8f5ec72d-fdf6-4153-a209-f28398f5380c None None] MFA Rules not processed for user `7634f750a07143a8b8d91cf0c8e78e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:19:39.399214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8f5ec72d-fdf6-4153-a209-f28398f5380c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:39.480207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1389/2801] 10.222.0.86 () {58 vars in 967 bytes} [Mon Aug 29 19:19:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:19:39.519114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:39.520811 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:39.614549 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085d319-1ae9-4e17-899f-bceeb0726572 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:39.619630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98df273-be86-40e4-9445-70f165d806bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1b7173843b4da48e69da8b36d59226', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:39.620231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98df273-be86-40e4-9445-70f165d806bc service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:39.620513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98df273-be86-40e4-9445-70f165d806bc service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:39.620789 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98df273-be86-40e4-9445-70f165d806bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:39.628717 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98df273-be86-40e4-9445-70f165d806bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:39.630291 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98df273-be86-40e4-9445-70f165d806bc service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:39.638246 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1413/2802] 10.222.0.86 () {60 vars in 1384 bytes} [Mon Aug 29 19:19:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:19:46.448700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ec72d-fdf6-4153-a209-f28398f5380c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:46.450326 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8f5ec72d-fdf6-4153-a209-f28398f5380c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:46.518687 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ec72d-fdf6-4153-a209-f28398f5380c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:46.524371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:46.524999 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:46.525271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:46.525579 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] PATH_INFO: `/v3/users/a4fe2e98ec3d42ccadd648a53fd7b074` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:46.526656 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4fe2e98ec3d42ccadd648a53fd7b074)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:46.528592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:46.586543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4fe2e98ec3d42ccadd648a53fd7b074'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:46.599823 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4fe2e98ec3d42ccadd648a53fd7b074'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:46.600897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4fe2e98ec3d42ccadd648a53fd7b074'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:46.601830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4fe2e98ec3d42ccadd648a53fd7b074'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:46.609287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4fe2e98ec3d42ccadd648a53fd7b074'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:46.610533 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:19:46.625767 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1390/2803] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:19:46 2022] DELETE /identity/v3/users/a4fe2e98ec3d42ccadd648a53fd7b074 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:19:46.806895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98df273-be86-40e4-9445-70f165d806bc service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:46.818125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98df273-be86-40e4-9445-70f165d806bc service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:46.875672 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:46.876293 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:46.876570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:46.876841 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] PATH_INFO: `/v3/projects/bd38972fd2dc4dda987f96f0a6aedcc7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:46.877934 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd38972fd2dc4dda987f96f0a6aedcc7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:46.879851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:46.927821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd38972fd2dc4dda987f96f0a6aedcc7', 'request_id': 'req-7131f208-4292-436c-910c-38272b93a31d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:46.935031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd38972fd2dc4dda987f96f0a6aedcc7', 'request_id': 'req-7131f208-4292-436c-910c-38272b93a31d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:46.941138 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd38972fd2dc4dda987f96f0a6aedcc7', 'request_id': 'req-7131f208-4292-436c-910c-38272b93a31d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:46.946617 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:19:46.954311 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] The token cache is being invalidate because project bd38972fd2dc4dda987f96f0a6aedcc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:19:46.954770 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:46.955794 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:46.957422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1414/2804] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:19:46 2022] DELETE /identity/v3/projects/bd38972fd2dc4dda987f96f0a6aedcc7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:19:46.961638 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:46.963630 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:46.976508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14359678-5e9c-4118-8eb3-55366b7eeba2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:47.031003 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8952e9-9428-42fd-ad5b-2c49a498ba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:47.031650 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8952e9-9428-42fd-ad5b-2c49a498ba23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:47.031928 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8952e9-9428-42fd-ad5b-2c49a498ba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:47.032198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8952e9-9428-42fd-ad5b-2c49a498ba23 admin admin] PATH_INFO: `/v3/users/e30fac9da61c4d06b0008e445ad7eaa9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:47.033312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8952e9-9428-42fd-ad5b-2c49a498ba23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e30fac9da61c4d06b0008e445ad7eaa9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:47.035245 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8952e9-9428-42fd-ad5b-2c49a498ba23 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:47.079520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1c8952e9-9428-42fd-ad5b-2c49a498ba23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e30fac9da61c4d06b0008e445ad7eaa9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:47.092258 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1c8952e9-9428-42fd-ad5b-2c49a498ba23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e30fac9da61c4d06b0008e445ad7eaa9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:47.092911 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1c8952e9-9428-42fd-ad5b-2c49a498ba23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e30fac9da61c4d06b0008e445ad7eaa9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:47.093520 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1c8952e9-9428-42fd-ad5b-2c49a498ba23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e30fac9da61c4d06b0008e445ad7eaa9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:47.100900 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1c8952e9-9428-42fd-ad5b-2c49a498ba23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e30fac9da61c4d06b0008e445ad7eaa9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:47.113062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1391/2805] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:19:46 2022] DELETE /identity/v3/users/e30fac9da61c4d06b0008e445ad7eaa9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:19:47.303420 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:47.305628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:47.321918 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131f208-4292-436c-910c-38272b93a31d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:47.383351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:47.384012 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:47.384259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:47.384517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] PATH_INFO: `/v3/projects/4055ec1f68d5433db9f16810d68b992b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:47.385641 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4055ec1f68d5433db9f16810d68b992b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:47.387547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:47.437607 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4055ec1f68d5433db9f16810d68b992b', 'request_id': 'req-54c42de7-8c88-49ea-a923-b4f22e6a357c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:47.446730 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4055ec1f68d5433db9f16810d68b992b', 'request_id': 'req-54c42de7-8c88-49ea-a923-b4f22e6a357c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:47.453443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4055ec1f68d5433db9f16810d68b992b', 'request_id': 'req-54c42de7-8c88-49ea-a923-b4f22e6a357c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:47.464343 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] The token cache is being invalidate because project 4055ec1f68d5433db9f16810d68b992b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:19:47.464714 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:47.465377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:47.466862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1415/2806] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:19:47 2022] DELETE /identity/v3/projects/4055ec1f68d5433db9f16810d68b992b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:19:55.256337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89d4b8-5c6a-45fe-be37-15c07467c5ea None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:55.256882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89d4b8-5c6a-45fe-be37-15c07467c5ea None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:55.256882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89d4b8-5c6a-45fe-be37-15c07467c5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:55.278085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1a89d4b8-5c6a-45fe-be37-15c07467c5ea None None] MFA Rules not processed for user `a982dbba4a324a4f893e427000ce9b93`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:19:55.279309 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1a89d4b8-5c6a-45fe-be37-15c07467c5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:55.332800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1392/2807] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:19:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:19:57.461123 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:19:58.656726 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:58.663669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:58.777469 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c42de7-8c88-49ea-a923-b4f22e6a357c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:58.783581 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349e743-f9d2-4660-8d0d-4177506222eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '045b22d046c549f2b53d2ed73b94f936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:58.784265 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349e743-f9d2-4660-8d0d-4177506222eb service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:58.784548 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349e743-f9d2-4660-8d0d-4177506222eb service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:58.784838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349e743-f9d2-4660-8d0d-4177506222eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:58.786809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3349e743-f9d2-4660-8d0d-4177506222eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:58.799664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349e743-f9d2-4660-8d0d-4177506222eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:58.801259 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349e743-f9d2-4660-8d0d-4177506222eb service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:58.812699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1416/2808] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:19:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:05.325772 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:08.808071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=43 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:20:47.974755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89d4b8-5c6a-45fe-be37-15c07467c5ea None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:20:47.976327 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1a89d4b8-5c6a-45fe-be37-15c07467c5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:48.067019 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89d4b8-5c6a-45fe-be37-15c07467c5ea None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:20:48.073031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e34fba-cd46-4b98-a5e6-db197acd757b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:20:48.073646 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e34fba-cd46-4b98-a5e6-db197acd757b service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:20:48.073929 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e34fba-cd46-4b98-a5e6-db197acd757b service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:20:48.074203 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e34fba-cd46-4b98-a5e6-db197acd757b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:20:48.075982 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-68e34fba-cd46-4b98-a5e6-db197acd757b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:20:48.088344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e34fba-cd46-4b98-a5e6-db197acd757b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:20:48.089918 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e34fba-cd46-4b98-a5e6-db197acd757b service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:20:48.099878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1393/2809] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:20:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:20:58.095560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=41 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:44.742450 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349e743-f9d2-4660-8d0d-4177506222eb service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:44.744434 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3349e743-f9d2-4660-8d0d-4177506222eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:44.820907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349e743-f9d2-4660-8d0d-4177506222eb service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:44.824989 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2e71f-f564-4bff-8c56-810c76582a8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:44.825585 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2e71f-f564-4bff-8c56-810c76582a8b service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:44.825864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2e71f-f564-4bff-8c56-810c76582a8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:44.826134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2e71f-f564-4bff-8c56-810c76582a8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:44.827947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6dc2e71f-f564-4bff-8c56-810c76582a8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:44.841391 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2e71f-f564-4bff-8c56-810c76582a8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:44.842790 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2e71f-f564-4bff-8c56-810c76582a8b service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:44.853514 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1417/2810] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:47.482374 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e34fba-cd46-4b98-a5e6-db197acd757b service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:47.483818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-68e34fba-cd46-4b98-a5e6-db197acd757b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:47.560363 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e34fba-cd46-4b98-a5e6-db197acd757b service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:47.563969 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05c675-9c56-4756-8df0-1e0556b68b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:47.564577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05c675-9c56-4756-8df0-1e0556b68b58 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:47.565149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05c675-9c56-4756-8df0-1e0556b68b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:47.565149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05c675-9c56-4756-8df0-1e0556b68b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:47.566801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ce05c675-9c56-4756-8df0-1e0556b68b58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:47.580186 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05c675-9c56-4756-8df0-1e0556b68b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:47.581574 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05c675-9c56-4756-8df0-1e0556b68b58 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:47.592778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1394/2811] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:54.848660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=35 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:55.487172 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2e71f-f564-4bff-8c56-810c76582a8b service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:55.496594 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2e71f-f564-4bff-8c56-810c76582a8b service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:55.500154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14175a-4ebe-4a4b-bf2b-0d4be8edef76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:55.500772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14175a-4ebe-4a4b-bf2b-0d4be8edef76 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:55.501052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14175a-4ebe-4a4b-bf2b-0d4be8edef76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:55.501325 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14175a-4ebe-4a4b-bf2b-0d4be8edef76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:55.503452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5c14175a-4ebe-4a4b-bf2b-0d4be8edef76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:55.586954 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14175a-4ebe-4a4b-bf2b-0d4be8edef76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:55.588429 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14175a-4ebe-4a4b-bf2b-0d4be8edef76 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:55.607026 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1418/2812] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:57.587957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=35 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:58.752810 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05c675-9c56-4756-8df0-1e0556b68b58 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:58.754239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ce05c675-9c56-4756-8df0-1e0556b68b58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:58.768070 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05c675-9c56-4756-8df0-1e0556b68b58 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:58.773293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7a115-e9ec-473d-b5c1-9f179b077e42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:58.773934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7a115-e9ec-473d-b5c1-9f179b077e42 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:58.774214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7a115-e9ec-473d-b5c1-9f179b077e42 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:58.774652 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7a115-e9ec-473d-b5c1-9f179b077e42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:58.776626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f8c7a115-e9ec-473d-b5c1-9f179b077e42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:58.788743 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7a115-e9ec-473d-b5c1-9f179b077e42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:58.791421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7a115-e9ec-473d-b5c1-9f179b077e42 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:58.803648 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1395/2813] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:21:58.809321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14175a-4ebe-4a4b-bf2b-0d4be8edef76 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:58.810874 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5c14175a-4ebe-4a4b-bf2b-0d4be8edef76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:58.823628 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14175a-4ebe-4a4b-bf2b-0d4be8edef76 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:58.832865 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc5b44-2fbf-4394-ae90-6963bb975608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:58.833497 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc5b44-2fbf-4394-ae90-6963bb975608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:58.833772 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc5b44-2fbf-4394-ae90-6963bb975608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:58.834042 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc5b44-2fbf-4394-ae90-6963bb975608 admin admin] PATH_INFO: `/v3/users/a982dbba4a324a4f893e427000ce9b93` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:58.835375 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc5b44-2fbf-4394-ae90-6963bb975608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a982dbba4a324a4f893e427000ce9b93)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:58.837448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc5b44-2fbf-4394-ae90-6963bb975608 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:58.882645 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-28cc5b44-2fbf-4394-ae90-6963bb975608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a982dbba4a324a4f893e427000ce9b93'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:58.883847 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-28cc5b44-2fbf-4394-ae90-6963bb975608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a982dbba4a324a4f893e427000ce9b93'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:58.895847 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-28cc5b44-2fbf-4394-ae90-6963bb975608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a982dbba4a324a4f893e427000ce9b93'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:58.903738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-28cc5b44-2fbf-4394-ae90-6963bb975608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a982dbba4a324a4f893e427000ce9b93'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:58.910890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-28cc5b44-2fbf-4394-ae90-6963bb975608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a982dbba4a324a4f893e427000ce9b93'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:58.916802 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1419/2814] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:21:58 2022] DELETE /identity/v3/users/a982dbba4a324a4f893e427000ce9b93 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:59.104183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7a115-e9ec-473d-b5c1-9f179b077e42 service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.115029 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7a115-e9ec-473d-b5c1-9f179b077e42 service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.174914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.175500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.175783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.176056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] PATH_INFO: `/v3/projects/fe9fc68075be4712a305fb8a15c91f19` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.177166 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9fc68075be4712a305fb8a15c91f19)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.179096 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.224895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9fc68075be4712a305fb8a15c91f19', 'request_id': 'req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:59.230961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9fc68075be4712a305fb8a15c91f19', 'request_id': 'req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:59.236641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9fc68075be4712a305fb8a15c91f19', 'request_id': 'req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:59.249185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] The token cache is being invalidate because project fe9fc68075be4712a305fb8a15c91f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:21:59.249471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:59.250092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3630cf6f-10ab-4d17-866b-f7d8cfbb2f3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:21:59.251577 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1396/2815] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:21:59 2022] DELETE /identity/v3/projects/fe9fc68075be4712a305fb8a15c91f19 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:59.256652 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b22c07-2519-4b36-831e-6ed745166724 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.256948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b22c07-2519-4b36-831e-6ed745166724 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.257255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b22c07-2519-4b36-831e-6ed745166724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.312134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:59.315561 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-19b22c07-2519-4b36-831e-6ed745166724 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:59.317435 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-19b22c07-2519-4b36-831e-6ed745166724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:59.380335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1420/2816] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:59.389131 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3fd91-f818-4713-97d9-317dcf1a8635 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.389417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3fd91-f818-4713-97d9-317dcf1a8635 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.389688 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3fd91-f818-4713-97d9-317dcf1a8635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.408808 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-29e3fd91-f818-4713-97d9-317dcf1a8635 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:59.409992 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-29e3fd91-f818-4713-97d9-317dcf1a8635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:59.418301 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1397/2817] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:59.422395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b22c07-2519-4b36-831e-6ed745166724 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.432964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b22c07-2519-4b36-831e-6ed745166724 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.437482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb4897-6b5c-4bd8-b810-4d4b3fb9b9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.438076 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb4897-6b5c-4bd8-b810-4d4b3fb9b9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.438362 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb4897-6b5c-4bd8-b810-4d4b3fb9b9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.438635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb4897-6b5c-4bd8-b810-4d4b3fb9b9f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.439317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb4897-6b5c-4bd8-b810-4d4b3fb9b9f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:21:59.439622 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb4897-6b5c-4bd8-b810-4d4b3fb9b9f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.441201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb4897-6b5c-4bd8-b810-4d4b3fb9b9f0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.452682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1421/2818] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:21:59.456855 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3fd91-f818-4713-97d9-317dcf1a8635 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.466181 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3fd91-f818-4713-97d9-317dcf1a8635 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.470732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87620f-138a-49a2-818f-f667c1f49181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.471349 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87620f-138a-49a2-818f-f667c1f49181 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.471621 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87620f-138a-49a2-818f-f667c1f49181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.471887 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87620f-138a-49a2-818f-f667c1f49181 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.472501 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87620f-138a-49a2-818f-f667c1f49181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.474387 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87620f-138a-49a2-818f-f667c1f49181 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.501785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1398/2819] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:59.505902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb4897-6b5c-4bd8-b810-4d4b3fb9b9f0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.516194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb4897-6b5c-4bd8-b810-4d4b3fb9b9f0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.574098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65627306-0900-4e22-8820-45324c3ccace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.574698 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65627306-0900-4e22-8820-45324c3ccace admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.574979 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65627306-0900-4e22-8820-45324c3ccace admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.575278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65627306-0900-4e22-8820-45324c3ccace admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.575902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65627306-0900-4e22-8820-45324c3ccace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.577800 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65627306-0900-4e22-8820-45324c3ccace admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.599340 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1422/2820] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:21:59.603513 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87620f-138a-49a2-818f-f667c1f49181 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.613713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87620f-138a-49a2-818f-f667c1f49181 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.618358 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65b2bc-5377-464c-8b63-598b7b36da80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.618949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b2bc-5377-464c-8b63-598b7b36da80 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.619266 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b2bc-5377-464c-8b63-598b7b36da80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.619539 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b2bc-5377-464c-8b63-598b7b36da80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.620174 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65b2bc-5377-464c-8b63-598b7b36da80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.621774 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65b2bc-5377-464c-8b63-598b7b36da80 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.631562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1399/2821] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:59.635963 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65627306-0900-4e22-8820-45324c3ccace admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.645975 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65627306-0900-4e22-8820-45324c3ccace admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.651273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b365eb5-70c3-4e21-8804-97cd504e3580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.651863 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b365eb5-70c3-4e21-8804-97cd504e3580 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.652143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b365eb5-70c3-4e21-8804-97cd504e3580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.652414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b365eb5-70c3-4e21-8804-97cd504e3580 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.653077 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b365eb5-70c3-4e21-8804-97cd504e3580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.654686 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b365eb5-70c3-4e21-8804-97cd504e3580 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.786605 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1423/2822] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:21:59.790839 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65b2bc-5377-464c-8b63-598b7b36da80 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.801132 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65b2bc-5377-464c-8b63-598b7b36da80 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:59.806176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494325c4-ecf5-448c-ac85-a85bd362c5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:59.806810 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494325c4-ecf5-448c-ac85-a85bd362c5e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.807119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494325c4-ecf5-448c-ac85-a85bd362c5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.807399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494325c4-ecf5-448c-ac85-a85bd362c5e1 admin admin] PATH_INFO: `/v3/projects/835ed831a0534fd18073d8615487c968/users/2b4e96775aa14dbfa2f14dd91c42f930/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.822388 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494325c4-ecf5-448c-ac85-a85bd362c5e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835ed831a0534fd18073d8615487c968, user_id=2b4e96775aa14dbfa2f14dd91c42f930, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:59.825049 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494325c4-ecf5-448c-ac85-a85bd362c5e1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:59.837877 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1400/2823] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:21:59 2022] PUT /identity/v3/projects/835ed831a0534fd18073d8615487c968/users/2b4e96775aa14dbfa2f14dd91c42f930/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:21:59.848043 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df4be0-013d-401e-b7f3-aac48b3d7f7c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:59.848558 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df4be0-013d-401e-b7f3-aac48b3d7f7c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:59.848664 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df4be0-013d-401e-b7f3-aac48b3d7f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:59.875691 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a2df4be0-013d-401e-b7f3-aac48b3d7f7c None None] MFA Rules not processed for user `2b4e96775aa14dbfa2f14dd91c42f930`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:21:59.877029 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a2df4be0-013d-401e-b7f3-aac48b3d7f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:59.938768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1424/2824] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:21:59.943332 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494325c4-ecf5-448c-ac85-a85bd362c5e1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:59.956862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:00.005207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494325c4-ecf5-448c-ac85-a85bd362c5e1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:00.010401 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547d36f-753a-4852-b596-61fc5674cda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:00.011017 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547d36f-753a-4852-b596-61fc5674cda1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:00.011312 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547d36f-753a-4852-b596-61fc5674cda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:00.011586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547d36f-753a-4852-b596-61fc5674cda1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.012219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547d36f-753a-4852-b596-61fc5674cda1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:00.014304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547d36f-753a-4852-b596-61fc5674cda1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:00.040287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1401/2825] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:21:59 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:22:00.044367 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df4be0-013d-401e-b7f3-aac48b3d7f7c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:00.049768 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:00.057104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df4be0-013d-401e-b7f3-aac48b3d7f7c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:00.112138 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff8fff-b9ec-435d-8eb7-cf3062614569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:00.112743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff8fff-b9ec-435d-8eb7-cf3062614569 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:00.113021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff8fff-b9ec-435d-8eb7-cf3062614569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:00.113288 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff8fff-b9ec-435d-8eb7-cf3062614569 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.113905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff8fff-b9ec-435d-8eb7-cf3062614569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:00.115818 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff8fff-b9ec-435d-8eb7-cf3062614569 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:00.136927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1425/2826] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:00 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:00.140948 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547d36f-753a-4852-b596-61fc5674cda1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:00.151471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547d36f-753a-4852-b596-61fc5674cda1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:00.156147 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67877a-aa12-4b71-9c8a-4aa30ebd1c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:00.156740 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67877a-aa12-4b71-9c8a-4aa30ebd1c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:00.157020 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67877a-aa12-4b71-9c8a-4aa30ebd1c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:00.157292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67877a-aa12-4b71-9c8a-4aa30ebd1c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.157926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67877a-aa12-4b71-9c8a-4aa30ebd1c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:00.159608 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67877a-aa12-4b71-9c8a-4aa30ebd1c2b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:00.170970 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1402/2827] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:00.175182 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff8fff-b9ec-435d-8eb7-cf3062614569 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:00.187062 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff8fff-b9ec-435d-8eb7-cf3062614569 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:00.191843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ed365-4e4a-4196-bba0-31a203295f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:00.192486 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ed365-4e4a-4196-bba0-31a203295f78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:00.192766 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ed365-4e4a-4196-bba0-31a203295f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:00.193033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ed365-4e4a-4196-bba0-31a203295f78 admin admin] PATH_INFO: `/v3/projects/374f6d299bed4ede886ccccd8ff4454b/users/cc2dc6e52b2343529fa0c420f679679f/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.208192 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ed365-4e4a-4196-bba0-31a203295f78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=374f6d299bed4ede886ccccd8ff4454b, user_id=cc2dc6e52b2343529fa0c420f679679f, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:00.210732 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ed365-4e4a-4196-bba0-31a203295f78 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:00.223977 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1426/2828] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:00 2022] PUT /identity/v3/projects/374f6d299bed4ede886ccccd8ff4454b/users/cc2dc6e52b2343529fa0c420f679679f/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:00.228495 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67877a-aa12-4b71-9c8a-4aa30ebd1c2b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:00.238408 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67877a-aa12-4b71-9c8a-4aa30ebd1c2b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:00.293843 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0840ab-07df-40ca-8b13-f81973c2b316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:00.294444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0840ab-07df-40ca-8b13-f81973c2b316 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:00.294726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0840ab-07df-40ca-8b13-f81973c2b316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:00.295014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0840ab-07df-40ca-8b13-f81973c2b316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.295660 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0840ab-07df-40ca-8b13-f81973c2b316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:00.297233 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0840ab-07df-40ca-8b13-f81973c2b316 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:00.307403 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1403/2829] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:00.311583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ed365-4e4a-4196-bba0-31a203295f78 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:00.321693 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ed365-4e4a-4196-bba0-31a203295f78 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:00.327071 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e1b98-7429-4f3d-87c0-412757dd903c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:00.327715 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e1b98-7429-4f3d-87c0-412757dd903c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:00.327993 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e1b98-7429-4f3d-87c0-412757dd903c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:00.328265 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e1b98-7429-4f3d-87c0-412757dd903c admin admin] PATH_INFO: `/v3/projects/374f6d299bed4ede886ccccd8ff4454b/users/cc2dc6e52b2343529fa0c420f679679f/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.330161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e1b98-7429-4f3d-87c0-412757dd903c admin admin] RBAC: Authorizing `identity:create_grant(project_id=374f6d299bed4ede886ccccd8ff4454b, user_id=cc2dc6e52b2343529fa0c420f679679f, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:00.333073 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e1b98-7429-4f3d-87c0-412757dd903c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:00.343133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1427/2830] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:00 2022] PUT /identity/v3/projects/374f6d299bed4ede886ccccd8ff4454b/users/cc2dc6e52b2343529fa0c420f679679f/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:00.355120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a882a-2147-4234-bb96-96bbaf4d2c42 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:00.355412 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a882a-2147-4234-bb96-96bbaf4d2c42 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:00.355711 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a882a-2147-4234-bb96-96bbaf4d2c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.383956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ce7a882a-2147-4234-bb96-96bbaf4d2c42 None None] MFA Rules not processed for user `cc2dc6e52b2343529fa0c420f679679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:00.385176 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ce7a882a-2147-4234-bb96-96bbaf4d2c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:00.451623 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1404/2831] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:00.464474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e1b98-7429-4f3d-87c0-412757dd903c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:00.466087 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-918e1b98-7429-4f3d-87c0-412757dd903c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:00.521744 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e1b98-7429-4f3d-87c0-412757dd903c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:00.525923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d4a43-4c85-4c6a-8747-d733ca1a78f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:00.526517 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d4a43-4c85-4c6a-8747-d733ca1a78f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:00.526796 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d4a43-4c85-4c6a-8747-d733ca1a78f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:00.527093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d4a43-4c85-4c6a-8747-d733ca1a78f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:00.535928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d4a43-4c85-4c6a-8747-d733ca1a78f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:00.537324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d4a43-4c85-4c6a-8747-d733ca1a78f1 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:00.547304 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1428/2832] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:00.931661 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a882a-2147-4234-bb96-96bbaf4d2c42 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:00.940995 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:00.997980 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a882a-2147-4234-bb96-96bbaf4d2c42 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:01.002582 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:01.003244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:01.003522 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:01.003791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] PATH_INFO: `/v3/users/2b4e96775aa14dbfa2f14dd91c42f930` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:01.004934 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4e96775aa14dbfa2f14dd91c42f930)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:01.006852 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:01.053828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4e96775aa14dbfa2f14dd91c42f930'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.065890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4e96775aa14dbfa2f14dd91c42f930'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.066540 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4e96775aa14dbfa2f14dd91c42f930'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.067157 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4e96775aa14dbfa2f14dd91c42f930'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.075036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4e96775aa14dbfa2f14dd91c42f930'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.088068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1405/2833] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:00 2022] DELETE /identity/v3/users/2b4e96775aa14dbfa2f14dd91c42f930 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:01.092577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c64a4-fb03-45b6-a613-80339574e967 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:01.093055 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c64a4-fb03-45b6-a613-80339574e967 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:01.093173 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c64a4-fb03-45b6-a613-80339574e967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:01.111673 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-123c64a4-fb03-45b6-a613-80339574e967 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:01.112872 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-123c64a4-fb03-45b6-a613-80339574e967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:01.161326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:01.171496 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1429/2834] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:01.181208 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:01.182758 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:01.240543 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804d92c-8c66-4f92-80db-120e65fd51de admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:01.244702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f92f32-df4b-43ba-b3be-05abc17d1425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:01.245295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f92f32-df4b-43ba-b3be-05abc17d1425 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:01.245571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f92f32-df4b-43ba-b3be-05abc17d1425 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:01.245840 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f92f32-df4b-43ba-b3be-05abc17d1425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:01.255249 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f92f32-df4b-43ba-b3be-05abc17d1425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:01.256616 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f92f32-df4b-43ba-b3be-05abc17d1425 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:01.266541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1406/2835] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:01.535970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c64a4-fb03-45b6-a613-80339574e967 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:01.537500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-123c64a4-fb03-45b6-a613-80339574e967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:01.551145 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c64a4-fb03-45b6-a613-80339574e967 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:01.555743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:01.556380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:01.556660 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:01.556927 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] PATH_INFO: `/v3/projects/835ed831a0534fd18073d8615487c968` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:01.558044 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] RBAC: Authorizing `identity:delete_project(project_id=835ed831a0534fd18073d8615487c968)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:01.559980 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:01.609035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835ed831a0534fd18073d8615487c968', 'request_id': 'req-f36424d5-a013-4ed5-a02e-838a8964762c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.616114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835ed831a0534fd18073d8615487c968', 'request_id': 'req-f36424d5-a013-4ed5-a02e-838a8964762c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.622354 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835ed831a0534fd18073d8615487c968', 'request_id': 'req-f36424d5-a013-4ed5-a02e-838a8964762c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.632773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] The token cache is being invalidate because project 835ed831a0534fd18073d8615487c968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:01.634133 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.634764 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.636243 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1430/2836] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:01 2022] DELETE /identity/v3/projects/835ed831a0534fd18073d8615487c968 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:01.640216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f92f32-df4b-43ba-b3be-05abc17d1425 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:01.650300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f92f32-df4b-43ba-b3be-05abc17d1425 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:01.678104 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:01.708949 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b154af2-e1bf-4e96-84dc-aaf2dda6bec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:01.709583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b154af2-e1bf-4e96-84dc-aaf2dda6bec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:01.709889 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b154af2-e1bf-4e96-84dc-aaf2dda6bec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:01.710169 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b154af2-e1bf-4e96-84dc-aaf2dda6bec6 admin admin] PATH_INFO: `/v3/users/cc2dc6e52b2343529fa0c420f679679f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:01.711895 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b154af2-e1bf-4e96-84dc-aaf2dda6bec6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2dc6e52b2343529fa0c420f679679f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:01.713854 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b154af2-e1bf-4e96-84dc-aaf2dda6bec6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:01.757713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3b154af2-e1bf-4e96-84dc-aaf2dda6bec6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2dc6e52b2343529fa0c420f679679f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.770016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3b154af2-e1bf-4e96-84dc-aaf2dda6bec6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2dc6e52b2343529fa0c420f679679f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.770654 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3b154af2-e1bf-4e96-84dc-aaf2dda6bec6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2dc6e52b2343529fa0c420f679679f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.771286 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3b154af2-e1bf-4e96-84dc-aaf2dda6bec6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2dc6e52b2343529fa0c420f679679f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.778796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3b154af2-e1bf-4e96-84dc-aaf2dda6bec6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2dc6e52b2343529fa0c420f679679f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:01.790708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1407/2837] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:01 2022] DELETE /identity/v3/users/cc2dc6e52b2343529fa0c420f679679f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:02.061482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.062983 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:02.079272 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36424d5-a013-4ed5-a02e-838a8964762c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.137113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.137870 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.138007 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.138273 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] PATH_INFO: `/v3/projects/374f6d299bed4ede886ccccd8ff4454b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.139545 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=374f6d299bed4ede886ccccd8ff4454b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.141299 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.189335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '374f6d299bed4ede886ccccd8ff4454b', 'request_id': 'req-a7511091-1b5b-435b-a822-3e46721f6a1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.196633 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '374f6d299bed4ede886ccccd8ff4454b', 'request_id': 'req-a7511091-1b5b-435b-a822-3e46721f6a1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.204309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '374f6d299bed4ede886ccccd8ff4454b', 'request_id': 'req-a7511091-1b5b-435b-a822-3e46721f6a1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.216035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] The token cache is being invalidate because project 374f6d299bed4ede886ccccd8ff4454b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:02.216319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.216974 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a7511091-1b5b-435b-a822-3e46721f6a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:02.218750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1431/2838] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:02 2022] DELETE /identity/v3/projects/374f6d299bed4ede886ccccd8ff4454b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:02.224351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55872-8d13-49c8-8073-0d24fb2fc1a8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.224952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55872-8d13-49c8-8073-0d24fb2fc1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.224952 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55872-8d13-49c8-8073-0d24fb2fc1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.245896 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3ad55872-8d13-49c8-8073-0d24fb2fc1a8 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:02.247174 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3ad55872-8d13-49c8-8073-0d24fb2fc1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:02.311156 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1408/2839] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:02.319261 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db207546-5a4a-4168-b39e-b305aaa077c5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.319642 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db207546-5a4a-4168-b39e-b305aaa077c5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.319916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db207546-5a4a-4168-b39e-b305aaa077c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.341788 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-db207546-5a4a-4168-b39e-b305aaa077c5 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:02.343947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db207546-5a4a-4168-b39e-b305aaa077c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:02.353181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1432/2840] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:02.357507 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55872-8d13-49c8-8073-0d24fb2fc1a8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.367503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55872-8d13-49c8-8073-0d24fb2fc1a8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.372343 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eb800-1aaf-4066-aa0c-86758e7c70ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.373085 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eb800-1aaf-4066-aa0c-86758e7c70ae admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.373461 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eb800-1aaf-4066-aa0c-86758e7c70ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.373742 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eb800-1aaf-4066-aa0c-86758e7c70ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.374393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78eb800-1aaf-4066-aa0c-86758e7c70ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:02.374702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78eb800-1aaf-4066-aa0c-86758e7c70ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.376346 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78eb800-1aaf-4066-aa0c-86758e7c70ae admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.389124 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1409/2841] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:02.394297 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db207546-5a4a-4168-b39e-b305aaa077c5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.400552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:02.404901 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db207546-5a4a-4168-b39e-b305aaa077c5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.412074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f357aa-4bbd-4e8a-a46b-725fa77c9116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.412908 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f357aa-4bbd-4e8a-a46b-725fa77c9116 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.413345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f357aa-4bbd-4e8a-a46b-725fa77c9116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.413684 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f357aa-4bbd-4e8a-a46b-725fa77c9116 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.414353 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f357aa-4bbd-4e8a-a46b-725fa77c9116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.416700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f357aa-4bbd-4e8a-a46b-725fa77c9116 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.444097 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1433/2842] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:02 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:02.448833 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eb800-1aaf-4066-aa0c-86758e7c70ae admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.458933 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eb800-1aaf-4066-aa0c-86758e7c70ae admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.493672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:02.519479 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b8444-84eb-4c14-9474-a0a3883529b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.520119 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b8444-84eb-4c14-9474-a0a3883529b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.520399 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b8444-84eb-4c14-9474-a0a3883529b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.520669 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b8444-84eb-4c14-9474-a0a3883529b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.521292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b8444-84eb-4c14-9474-a0a3883529b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.523220 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b8444-84eb-4c14-9474-a0a3883529b2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.543939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1410/2843] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:02 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:02.548104 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f357aa-4bbd-4e8a-a46b-725fa77c9116 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.558944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f357aa-4bbd-4e8a-a46b-725fa77c9116 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.563616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a0ff0-bade-4370-bf42-5b2a164ce31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.564152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a0ff0-bade-4370-bf42-5b2a164ce31f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.564431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a0ff0-bade-4370-bf42-5b2a164ce31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.564703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a0ff0-bade-4370-bf42-5b2a164ce31f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.565453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a0ff0-bade-4370-bf42-5b2a164ce31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.567099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a0ff0-bade-4370-bf42-5b2a164ce31f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.576851 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1434/2844] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:22:02.581036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b8444-84eb-4c14-9474-a0a3883529b2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.592236 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b8444-84eb-4c14-9474-a0a3883529b2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.596998 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e326d6-b356-4f6e-99a1-c3ffa049fc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.597594 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e326d6-b356-4f6e-99a1-c3ffa049fc5f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.597875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e326d6-b356-4f6e-99a1-c3ffa049fc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.598151 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e326d6-b356-4f6e-99a1-c3ffa049fc5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.598785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e326d6-b356-4f6e-99a1-c3ffa049fc5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.600397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e326d6-b356-4f6e-99a1-c3ffa049fc5f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.610664 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1411/2845] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:02.614960 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a0ff0-bade-4370-bf42-5b2a164ce31f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.625413 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a0ff0-bade-4370-bf42-5b2a164ce31f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.629993 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66010b-166c-4284-99d7-8d9190ec7afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.630627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66010b-166c-4284-99d7-8d9190ec7afe admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.630905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66010b-166c-4284-99d7-8d9190ec7afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.631203 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66010b-166c-4284-99d7-8d9190ec7afe admin admin] PATH_INFO: `/v3/projects/fdca964e043b4ace9191dbc0f4d79856/users/865e0917fbf74f7e91f4bf5daaab48e6/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.645216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66010b-166c-4284-99d7-8d9190ec7afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdca964e043b4ace9191dbc0f4d79856, user_id=865e0917fbf74f7e91f4bf5daaab48e6, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.647881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66010b-166c-4284-99d7-8d9190ec7afe admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.660697 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1435/2846] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:02 2022] PUT /identity/v3/projects/fdca964e043b4ace9191dbc0f4d79856/users/865e0917fbf74f7e91f4bf5daaab48e6/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:02.671791 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40139887-2b58-43cb-a545-3129947bd639 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.672078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40139887-2b58-43cb-a545-3129947bd639 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.672351 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40139887-2b58-43cb-a545-3129947bd639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.701559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-40139887-2b58-43cb-a545-3129947bd639 None None] MFA Rules not processed for user `865e0917fbf74f7e91f4bf5daaab48e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:02.702778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-40139887-2b58-43cb-a545-3129947bd639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:02.766446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1412/2847] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:02.772890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66010b-166c-4284-99d7-8d9190ec7afe admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.835221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66010b-166c-4284-99d7-8d9190ec7afe admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.839827 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da035be-4b3c-49e2-a7f3-6669fc5157ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.840423 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da035be-4b3c-49e2-a7f3-6669fc5157ff admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.840703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da035be-4b3c-49e2-a7f3-6669fc5157ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.840972 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da035be-4b3c-49e2-a7f3-6669fc5157ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.841609 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da035be-4b3c-49e2-a7f3-6669fc5157ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.843579 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da035be-4b3c-49e2-a7f3-6669fc5157ff admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.870471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1436/2848] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:02 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:02.874792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40139887-2b58-43cb-a545-3129947bd639 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.886322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40139887-2b58-43cb-a545-3129947bd639 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.939663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630acda-01ac-4c84-8716-1f71f9297b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.940253 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630acda-01ac-4c84-8716-1f71f9297b44 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.940530 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630acda-01ac-4c84-8716-1f71f9297b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.940797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630acda-01ac-4c84-8716-1f71f9297b44 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.941413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630acda-01ac-4c84-8716-1f71f9297b44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.943322 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630acda-01ac-4c84-8716-1f71f9297b44 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.964499 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1413/2849] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:02 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:22:02.968495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da035be-4b3c-49e2-a7f3-6669fc5157ff admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:02.979020 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da035be-4b3c-49e2-a7f3-6669fc5157ff admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:02.983556 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65121d1-0d55-4e05-91d3-0af1d76d9fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:02.984154 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65121d1-0d55-4e05-91d3-0af1d76d9fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:02.984431 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65121d1-0d55-4e05-91d3-0af1d76d9fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:02.984699 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65121d1-0d55-4e05-91d3-0af1d76d9fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:02.985329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65121d1-0d55-4e05-91d3-0af1d76d9fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:02.986898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65121d1-0d55-4e05-91d3-0af1d76d9fdf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:02.997843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1437/2850] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:03.001793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630acda-01ac-4c84-8716-1f71f9297b44 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:03.012428 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630acda-01ac-4c84-8716-1f71f9297b44 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:03.016941 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03db36-0ccb-4845-a3b1-bb9b8bd3846c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:03.017579 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03db36-0ccb-4845-a3b1-bb9b8bd3846c admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:03.017860 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03db36-0ccb-4845-a3b1-bb9b8bd3846c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:03.018135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03db36-0ccb-4845-a3b1-bb9b8bd3846c admin admin] PATH_INFO: `/v3/projects/d260e09576f8478181b1a744d2139c98/users/3d1f4e73d37943d5b84e2643b35c25ad/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:03.032378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03db36-0ccb-4845-a3b1-bb9b8bd3846c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d260e09576f8478181b1a744d2139c98, user_id=3d1f4e73d37943d5b84e2643b35c25ad, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:03.035042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03db36-0ccb-4845-a3b1-bb9b8bd3846c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:03.047519 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1414/2851] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:03 2022] PUT /identity/v3/projects/d260e09576f8478181b1a744d2139c98/users/3d1f4e73d37943d5b84e2643b35c25ad/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:03.051422 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65121d1-0d55-4e05-91d3-0af1d76d9fdf admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:03.061666 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65121d1-0d55-4e05-91d3-0af1d76d9fdf admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:03.117175 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c90a9-3f61-48b1-8804-09f2a85803a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:03.118231 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c90a9-3f61-48b1-8804-09f2a85803a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:03.118506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c90a9-3f61-48b1-8804-09f2a85803a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:03.118771 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c90a9-3f61-48b1-8804-09f2a85803a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:03.119419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c90a9-3f61-48b1-8804-09f2a85803a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:03.120979 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c90a9-3f61-48b1-8804-09f2a85803a0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:03.130483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1438/2852] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:03.134592 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03db36-0ccb-4845-a3b1-bb9b8bd3846c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:03.144824 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03db36-0ccb-4845-a3b1-bb9b8bd3846c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:03.149379 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e29440-739f-4678-94be-60d36336e9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:03.150021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e29440-739f-4678-94be-60d36336e9c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:03.150299 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e29440-739f-4678-94be-60d36336e9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:03.150597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e29440-739f-4678-94be-60d36336e9c4 admin admin] PATH_INFO: `/v3/projects/d260e09576f8478181b1a744d2139c98/users/3d1f4e73d37943d5b84e2643b35c25ad/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:03.152541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e29440-739f-4678-94be-60d36336e9c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d260e09576f8478181b1a744d2139c98, user_id=3d1f4e73d37943d5b84e2643b35c25ad, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:03.155188 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e29440-739f-4678-94be-60d36336e9c4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:03.160610 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:03.166693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1415/2853] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:03 2022] PUT /identity/v3/projects/d260e09576f8478181b1a744d2139c98/users/3d1f4e73d37943d5b84e2643b35c25ad/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:03.177735 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d673c4-6cf9-44fe-ae49-023f6fcfec21 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:03.178239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d673c4-6cf9-44fe-ae49-023f6fcfec21 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:03.178355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d673c4-6cf9-44fe-ae49-023f6fcfec21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:03.206506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b9d673c4-6cf9-44fe-ae49-023f6fcfec21 None None] MFA Rules not processed for user `3d1f4e73d37943d5b84e2643b35c25ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:03.207985 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b9d673c4-6cf9-44fe-ae49-023f6fcfec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:03.213129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:03.277204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1439/2854] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:03.287878 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e29440-739f-4678-94be-60d36336e9c4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:03.290092 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e6e29440-739f-4678-94be-60d36336e9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:03.347597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e29440-739f-4678-94be-60d36336e9c4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:03.351216 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2de58-530b-4333-b9d7-93d795074371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:03.352334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2de58-530b-4333-b9d7-93d795074371 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:03.352597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2de58-530b-4333-b9d7-93d795074371 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:03.352869 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2de58-530b-4333-b9d7-93d795074371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:03.406613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2de58-530b-4333-b9d7-93d795074371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:03.407985 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2de58-530b-4333-b9d7-93d795074371 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:03.417411 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1416/2855] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:10.096703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d673c4-6cf9-44fe-ae49-023f6fcfec21 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:10.106372 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d673c4-6cf9-44fe-ae49-023f6fcfec21 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:10.109912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1a6b6-e2a1-459d-9914-85a492991fb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:10.110505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1a6b6-e2a1-459d-9914-85a492991fb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:10.110784 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1a6b6-e2a1-459d-9914-85a492991fb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:10.111091 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1a6b6-e2a1-459d-9914-85a492991fb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:10.119830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1a6b6-e2a1-459d-9914-85a492991fb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:10.121233 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1a6b6-e2a1-459d-9914-85a492991fb8 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:10.130619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1440/2856] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:13.412899 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=49 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:20.126455 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=32 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:39.100766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2de58-530b-4333-b9d7-93d795074371 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:39.164870 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2de58-530b-4333-b9d7-93d795074371 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:39.169444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:39.170072 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:39.170354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:39.170628 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] PATH_INFO: `/v3/users/865e0917fbf74f7e91f4bf5daaab48e6` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:39.171768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=865e0917fbf74f7e91f4bf5daaab48e6)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:39.173783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:39.219300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '865e0917fbf74f7e91f4bf5daaab48e6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.232371 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '865e0917fbf74f7e91f4bf5daaab48e6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.233054 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '865e0917fbf74f7e91f4bf5daaab48e6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.233632 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '865e0917fbf74f7e91f4bf5daaab48e6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.240732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '865e0917fbf74f7e91f4bf5daaab48e6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.253065 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1417/2857] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:39 2022] DELETE /identity/v3/users/865e0917fbf74f7e91f4bf5daaab48e6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:39.258269 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82d77b-6aaa-4829-bca5-b0876736919c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:39.258756 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82d77b-6aaa-4829-bca5-b0876736919c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:39.258890 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82d77b-6aaa-4829-bca5-b0876736919c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:39.277734 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3f82d77b-6aaa-4829-bca5-b0876736919c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:39.278939 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3f82d77b-6aaa-4829-bca5-b0876736919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:39.336201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1441/2858] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:39.345959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:39.347390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:39.404421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea7fcb-3fc6-49f9-b8ea-aec1678254a7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:39.408116 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea844f7-b616-429f-adb1-7da951944a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:39.408745 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea844f7-b616-429f-adb1-7da951944a76 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:39.409021 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea844f7-b616-429f-adb1-7da951944a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:39.409291 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea844f7-b616-429f-adb1-7da951944a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:39.418422 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea844f7-b616-429f-adb1-7da951944a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:39.419873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea844f7-b616-429f-adb1-7da951944a76 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:39.429673 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1418/2859] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:39.699627 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82d77b-6aaa-4829-bca5-b0876736919c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:39.701051 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3f82d77b-6aaa-4829-bca5-b0876736919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:39.714129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82d77b-6aaa-4829-bca5-b0876736919c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:39.718631 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:39.719290 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:39.719566 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:39.719885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] PATH_INFO: `/v3/projects/fdca964e043b4ace9191dbc0f4d79856` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:39.720982 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdca964e043b4ace9191dbc0f4d79856)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:39.722899 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:39.769347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdca964e043b4ace9191dbc0f4d79856', 'request_id': 'req-7625e6e1-95fd-4141-9c14-811606189576'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.775912 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdca964e043b4ace9191dbc0f4d79856', 'request_id': 'req-7625e6e1-95fd-4141-9c14-811606189576'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.782301 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdca964e043b4ace9191dbc0f4d79856', 'request_id': 'req-7625e6e1-95fd-4141-9c14-811606189576'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.793443 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] The token cache is being invalidate because project fdca964e043b4ace9191dbc0f4d79856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:39.793737 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.794400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.795898 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1442/2860] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:39 2022] DELETE /identity/v3/projects/fdca964e043b4ace9191dbc0f4d79856 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:39.799913 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea844f7-b616-429f-adb1-7da951944a76 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:39.812496 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea844f7-b616-429f-adb1-7da951944a76 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:39.831261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:39.871458 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275e88f-c757-47db-b942-bf8b5450e968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:39.872087 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275e88f-c757-47db-b942-bf8b5450e968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:39.872366 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275e88f-c757-47db-b942-bf8b5450e968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:39.872642 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275e88f-c757-47db-b942-bf8b5450e968 admin admin] PATH_INFO: `/v3/users/3d1f4e73d37943d5b84e2643b35c25ad` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:39.873746 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275e88f-c757-47db-b942-bf8b5450e968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d1f4e73d37943d5b84e2643b35c25ad)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:39.875783 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275e88f-c757-47db-b942-bf8b5450e968 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:39.921337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f275e88f-c757-47db-b942-bf8b5450e968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d1f4e73d37943d5b84e2643b35c25ad'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.932834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f275e88f-c757-47db-b942-bf8b5450e968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d1f4e73d37943d5b84e2643b35c25ad'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.933481 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f275e88f-c757-47db-b942-bf8b5450e968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d1f4e73d37943d5b84e2643b35c25ad'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.934103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f275e88f-c757-47db-b942-bf8b5450e968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d1f4e73d37943d5b84e2643b35c25ad'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.941088 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f275e88f-c757-47db-b942-bf8b5450e968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d1f4e73d37943d5b84e2643b35c25ad'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:39.953367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1419/2861] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:39 2022] DELETE /identity/v3/users/3d1f4e73d37943d5b84e2643b35c25ad => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:40.223464 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:40.224948 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:40.238339 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625e6e1-95fd-4141-9c14-811606189576 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:40.295286 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:40.295942 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:40.296186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:40.296466 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] PATH_INFO: `/v3/projects/d260e09576f8478181b1a744d2139c98` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.297576 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d260e09576f8478181b1a744d2139c98)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:40.299502 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:40.330004 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:40.347583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd260e09576f8478181b1a744d2139c98', 'request_id': 'req-4d4b782e-ed0c-4f0e-925e-c1664885582c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:40.355204 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd260e09576f8478181b1a744d2139c98', 'request_id': 'req-4d4b782e-ed0c-4f0e-925e-c1664885582c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:40.361515 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd260e09576f8478181b1a744d2139c98', 'request_id': 'req-4d4b782e-ed0c-4f0e-925e-c1664885582c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:40.374321 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] The token cache is being invalidate because project d260e09576f8478181b1a744d2139c98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:40.374616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:40.375659 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4d4b782e-ed0c-4f0e-925e-c1664885582c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:40.377438 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1443/2862] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:40 2022] DELETE /identity/v3/projects/d260e09576f8478181b1a744d2139c98 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:40.382319 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ea077-b964-442f-bc9e-39bf9b4df5c6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:40.382785 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ea077-b964-442f-bc9e-39bf9b4df5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:40.382939 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ea077-b964-442f-bc9e-39bf9b4df5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.402738 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-537ea077-b964-442f-bc9e-39bf9b4df5c6 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:40.403915 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-537ea077-b964-442f-bc9e-39bf9b4df5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:40.462796 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1420/2863] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:40.470550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6862dc-7963-44f1-b423-5b9314990543 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:40.470828 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6862dc-7963-44f1-b423-5b9314990543 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:40.471139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6862dc-7963-44f1-b423-5b9314990543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.491099 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ae6862dc-7963-44f1-b423-5b9314990543 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:40.492883 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ae6862dc-7963-44f1-b423-5b9314990543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:40.500833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1444/2864] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:40.505009 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ea077-b964-442f-bc9e-39bf9b4df5c6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:40.514500 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ea077-b964-442f-bc9e-39bf9b4df5c6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:40.519275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966b127-97a9-47a7-9382-1cd81b4ffddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:40.519874 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966b127-97a9-47a7-9382-1cd81b4ffddc admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:40.520153 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966b127-97a9-47a7-9382-1cd81b4ffddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:40.520423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966b127-97a9-47a7-9382-1cd81b4ffddc admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.521063 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966b127-97a9-47a7-9382-1cd81b4ffddc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:40.521392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966b127-97a9-47a7-9382-1cd81b4ffddc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:40.523017 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966b127-97a9-47a7-9382-1cd81b4ffddc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:40.534462 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1421/2865] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:40.539125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6862dc-7963-44f1-b423-5b9314990543 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:40.548255 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6862dc-7963-44f1-b423-5b9314990543 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:40.552848 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8876f-051a-4bad-a793-380388b87906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:40.553428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8876f-051a-4bad-a793-380388b87906 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:40.553703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8876f-051a-4bad-a793-380388b87906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:40.554495 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8876f-051a-4bad-a793-380388b87906 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.555163 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8876f-051a-4bad-a793-380388b87906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:40.557011 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8876f-051a-4bad-a793-380388b87906 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:40.584453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1445/2866] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:40 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:40.588680 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966b127-97a9-47a7-9382-1cd81b4ffddc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:40.598686 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966b127-97a9-47a7-9382-1cd81b4ffddc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:40.624950 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:40.656056 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7aefd-caa6-4fe3-aa7f-94701379ca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:40.656585 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7aefd-caa6-4fe3-aa7f-94701379ca6d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:40.656863 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7aefd-caa6-4fe3-aa7f-94701379ca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:40.657139 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7aefd-caa6-4fe3-aa7f-94701379ca6d admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.657759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7aefd-caa6-4fe3-aa7f-94701379ca6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:40.659687 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7aefd-caa6-4fe3-aa7f-94701379ca6d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:40.682062 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1422/2867] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:40 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:40.686331 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8876f-051a-4bad-a793-380388b87906 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:40.697316 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8876f-051a-4bad-a793-380388b87906 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:40.702031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cce5f-1bf3-492e-9c95-1ad79715708a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:40.702619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cce5f-1bf3-492e-9c95-1ad79715708a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:40.702893 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cce5f-1bf3-492e-9c95-1ad79715708a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:40.703201 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cce5f-1bf3-492e-9c95-1ad79715708a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.703830 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cce5f-1bf3-492e-9c95-1ad79715708a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:40.705472 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cce5f-1bf3-492e-9c95-1ad79715708a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:40.716747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1446/2868] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:40.720761 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7aefd-caa6-4fe3-aa7f-94701379ca6d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:40.730711 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7aefd-caa6-4fe3-aa7f-94701379ca6d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:40.735548 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0a34e-e9a4-4674-9432-c32ecc5533b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:40.736171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0a34e-e9a4-4674-9432-c32ecc5533b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:40.736448 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0a34e-e9a4-4674-9432-c32ecc5533b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:40.736719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0a34e-e9a4-4674-9432-c32ecc5533b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.737376 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0a34e-e9a4-4674-9432-c32ecc5533b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:40.738947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0a34e-e9a4-4674-9432-c32ecc5533b6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:40.749786 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1423/2869] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:40.753659 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cce5f-1bf3-492e-9c95-1ad79715708a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:40.764126 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cce5f-1bf3-492e-9c95-1ad79715708a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:40.768905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdf938-ed5b-45e0-9c9c-46529c5fe5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:40.769553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdf938-ed5b-45e0-9c9c-46529c5fe5f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:40.769833 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdf938-ed5b-45e0-9c9c-46529c5fe5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:40.770107 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdf938-ed5b-45e0-9c9c-46529c5fe5f2 admin admin] PATH_INFO: `/v3/projects/163396fda144470692ee70659433dde2/users/04a4d2bb164544509255bcb5b68929e8/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.784525 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bdf938-ed5b-45e0-9c9c-46529c5fe5f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163396fda144470692ee70659433dde2, user_id=04a4d2bb164544509255bcb5b68929e8, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:40.787207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bdf938-ed5b-45e0-9c9c-46529c5fe5f2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:40.800218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1447/2870] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:40 2022] PUT /identity/v3/projects/163396fda144470692ee70659433dde2/users/04a4d2bb164544509255bcb5b68929e8/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:40.865875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691df8e1-c078-49ac-8c7f-80ad61c00943 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:40.866155 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691df8e1-c078-49ac-8c7f-80ad61c00943 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:40.866923 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691df8e1-c078-49ac-8c7f-80ad61c00943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:40.896759 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-691df8e1-c078-49ac-8c7f-80ad61c00943 None None] MFA Rules not processed for user `04a4d2bb164544509255bcb5b68929e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:40.897957 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-691df8e1-c078-49ac-8c7f-80ad61c00943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:40.960396 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1424/2871] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:40.964911 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdf938-ed5b-45e0-9c9c-46529c5fe5f2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:41.027156 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdf938-ed5b-45e0-9c9c-46529c5fe5f2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:41.031738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ae45a-9e5d-4ce7-88eb-a40fdfd5e84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:41.032407 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ae45a-9e5d-4ce7-88eb-a40fdfd5e84c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.032682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ae45a-9e5d-4ce7-88eb-a40fdfd5e84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.032947 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ae45a-9e5d-4ce7-88eb-a40fdfd5e84c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.033567 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ae45a-9e5d-4ce7-88eb-a40fdfd5e84c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:41.035482 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ae45a-9e5d-4ce7-88eb-a40fdfd5e84c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:41.061189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1448/2872] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:40 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:41.065291 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691df8e1-c078-49ac-8c7f-80ad61c00943 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:41.076204 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691df8e1-c078-49ac-8c7f-80ad61c00943 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:41.131149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf24a0e-6da5-4a9c-af2b-f9c9c11bbc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:41.131750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf24a0e-6da5-4a9c-af2b-f9c9c11bbc7f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.132536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf24a0e-6da5-4a9c-af2b-f9c9c11bbc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.132804 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf24a0e-6da5-4a9c-af2b-f9c9c11bbc7f admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.133429 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf24a0e-6da5-4a9c-af2b-f9c9c11bbc7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:41.135341 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf24a0e-6da5-4a9c-af2b-f9c9c11bbc7f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:41.155818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1425/2873] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:41 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:41.161009 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ae45a-9e5d-4ce7-88eb-a40fdfd5e84c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:41.171432 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ae45a-9e5d-4ce7-88eb-a40fdfd5e84c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:41.176276 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d2437-1020-4241-a4e8-a8bb8dd5607c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:41.176862 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d2437-1020-4241-a4e8-a8bb8dd5607c admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.177143 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d2437-1020-4241-a4e8-a8bb8dd5607c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.177410 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d2437-1020-4241-a4e8-a8bb8dd5607c admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.178035 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d2437-1020-4241-a4e8-a8bb8dd5607c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:41.179635 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d2437-1020-4241-a4e8-a8bb8dd5607c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:41.189781 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1449/2874] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:41.193850 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf24a0e-6da5-4a9c-af2b-f9c9c11bbc7f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:41.204269 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf24a0e-6da5-4a9c-af2b-f9c9c11bbc7f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:41.209118 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec234ea2-9dee-4a5e-b5d0-ca59712abfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:41.209755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec234ea2-9dee-4a5e-b5d0-ca59712abfc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.210035 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec234ea2-9dee-4a5e-b5d0-ca59712abfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.211033 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec234ea2-9dee-4a5e-b5d0-ca59712abfc1 admin admin] PATH_INFO: `/v3/projects/556c774bedd942ce8f513b3cfa1b1be0/users/d7d675550088431dacebc11a337b7100/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.226089 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec234ea2-9dee-4a5e-b5d0-ca59712abfc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556c774bedd942ce8f513b3cfa1b1be0, user_id=d7d675550088431dacebc11a337b7100, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:41.228622 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec234ea2-9dee-4a5e-b5d0-ca59712abfc1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:41.240834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1426/2875] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:41 2022] PUT /identity/v3/projects/556c774bedd942ce8f513b3cfa1b1be0/users/d7d675550088431dacebc11a337b7100/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:41.244712 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d2437-1020-4241-a4e8-a8bb8dd5607c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:41.254529 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d2437-1020-4241-a4e8-a8bb8dd5607c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:41.274552 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:41.311478 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9e5c9-3fff-4f38-bc69-db679ef3ce43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:41.312142 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9e5c9-3fff-4f38-bc69-db679ef3ce43 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.312418 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9e5c9-3fff-4f38-bc69-db679ef3ce43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.312724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9e5c9-3fff-4f38-bc69-db679ef3ce43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.313451 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9e5c9-3fff-4f38-bc69-db679ef3ce43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:41.314933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9e5c9-3fff-4f38-bc69-db679ef3ce43 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:41.325094 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1450/2876] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:41.329274 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec234ea2-9dee-4a5e-b5d0-ca59712abfc1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:41.334890 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:41.341121 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec234ea2-9dee-4a5e-b5d0-ca59712abfc1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:41.345897 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8026c0b-f1b5-455b-87a8-d742bef604b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:41.346541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8026c0b-f1b5-455b-87a8-d742bef604b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.346821 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8026c0b-f1b5-455b-87a8-d742bef604b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.347130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8026c0b-f1b5-455b-87a8-d742bef604b5 admin admin] PATH_INFO: `/v3/projects/556c774bedd942ce8f513b3cfa1b1be0/users/d7d675550088431dacebc11a337b7100/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.349036 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8026c0b-f1b5-455b-87a8-d742bef604b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556c774bedd942ce8f513b3cfa1b1be0, user_id=d7d675550088431dacebc11a337b7100, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:41.352196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8026c0b-f1b5-455b-87a8-d742bef604b5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:41.362123 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1427/2877] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:41 2022] PUT /identity/v3/projects/556c774bedd942ce8f513b3cfa1b1be0/users/d7d675550088431dacebc11a337b7100/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:41.372457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c658d66-f007-451d-a50d-e427c7b0929e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.372746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c658d66-f007-451d-a50d-e427c7b0929e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.373021 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c658d66-f007-451d-a50d-e427c7b0929e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.400135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7c658d66-f007-451d-a50d-e427c7b0929e None None] MFA Rules not processed for user `d7d675550088431dacebc11a337b7100`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:41.401317 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7c658d66-f007-451d-a50d-e427c7b0929e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:41.467383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1451/2878] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:41.480361 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8026c0b-f1b5-455b-87a8-d742bef604b5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:41.482081 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b8026c0b-f1b5-455b-87a8-d742bef604b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:41.539600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8026c0b-f1b5-455b-87a8-d742bef604b5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:41.543335 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71c87a-9c5e-406d-a80c-0587690a13cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:41.543945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71c87a-9c5e-406d-a80c-0587690a13cc service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.544222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71c87a-9c5e-406d-a80c-0587690a13cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.544508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71c87a-9c5e-406d-a80c-0587690a13cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.555506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71c87a-9c5e-406d-a80c-0587690a13cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:41.557571 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71c87a-9c5e-406d-a80c-0587690a13cc service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:41.567227 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1428/2879] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:41.871618 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c658d66-f007-451d-a50d-e427c7b0929e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:41.880911 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c658d66-f007-451d-a50d-e427c7b0929e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:41.887650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d352a-478f-4d66-9d79-b95096da5e0f tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7d675550088431dacebc11a337b7100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '556c774bedd942ce8f513b3cfa1b1be0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:41.889023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d352a-478f-4d66-9d79-b95096da5e0f tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.889023 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d352a-478f-4d66-9d79-b95096da5e0f tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.889330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d352a-478f-4d66-9d79-b95096da5e0f tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.889979 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d352a-478f-4d66-9d79-b95096da5e0f tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:41.890302 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d352a-478f-4d66-9d79-b95096da5e0f tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:41.891930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d352a-478f-4d66-9d79-b95096da5e0f tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:41.904646 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1452/2880] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:41.909451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71c87a-9c5e-406d-a80c-0587690a13cc service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:41.919120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71c87a-9c5e-406d-a80c-0587690a13cc service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:41.923663 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecabb95-4c3b-4915-ac77-dcb4f0812a26 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7d675550088431dacebc11a337b7100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '556c774bedd942ce8f513b3cfa1b1be0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:41.924244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecabb95-4c3b-4915-ac77-dcb4f0812a26 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:41.924524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecabb95-4c3b-4915-ac77-dcb4f0812a26 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:41.924801 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecabb95-4c3b-4915-ac77-dcb4f0812a26 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:41.925569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecabb95-4c3b-4915-ac77-dcb4f0812a26 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:41.927475 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecabb95-4c3b-4915-ac77-dcb4f0812a26 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:41.955334 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1429/2881] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:41 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:41.959710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d352a-478f-4d66-9d79-b95096da5e0f tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:41.970098 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d352a-478f-4d66-9d79-b95096da5e0f tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:42.025176 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941d06b-fba4-4abd-969b-ec5ebc937284 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7d675550088431dacebc11a337b7100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '556c774bedd942ce8f513b3cfa1b1be0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:42.025769 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941d06b-fba4-4abd-969b-ec5ebc937284 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:42.026054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941d06b-fba4-4abd-969b-ec5ebc937284 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:42.026333 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941d06b-fba4-4abd-969b-ec5ebc937284 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:42.026984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941d06b-fba4-4abd-969b-ec5ebc937284 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:42.027322 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941d06b-fba4-4abd-969b-ec5ebc937284 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:42.028902 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941d06b-fba4-4abd-969b-ec5ebc937284 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:42.040577 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1453/2882] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:42.076754 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecabb95-4c3b-4915-ac77-dcb4f0812a26 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:42.088126 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecabb95-4c3b-4915-ac77-dcb4f0812a26 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:42.092655 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7d675550088431dacebc11a337b7100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '556c774bedd942ce8f513b3cfa1b1be0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:42.093275 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:42.093559 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:42.093834 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] PATH_INFO: `/v3/projects/0a7ea3dd683c45f2b7668e63bc77f4b2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:42.094955 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0a7ea3dd683c45f2b7668e63bc77f4b2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:42.096959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:42.141206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a7ea3dd683c45f2b7668e63bc77f4b2', 'request_id': 'req-68447890-fed5-4474-b5bb-4f9c95aa20ab'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:42.148274 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a7ea3dd683c45f2b7668e63bc77f4b2', 'request_id': 'req-68447890-fed5-4474-b5bb-4f9c95aa20ab'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:42.153774 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a7ea3dd683c45f2b7668e63bc77f4b2', 'request_id': 'req-68447890-fed5-4474-b5bb-4f9c95aa20ab'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:42.167180 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] The token cache is being invalidate because project 0a7ea3dd683c45f2b7668e63bc77f4b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:42.167470 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:42.168101 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-68447890-fed5-4474-b5bb-4f9c95aa20ab tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:42.169586 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1430/2883] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:42 2022] DELETE /identity/v3/projects/0a7ea3dd683c45f2b7668e63bc77f4b2 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:42.362135 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941d06b-fba4-4abd-969b-ec5ebc937284 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:42.364012 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a941d06b-fba4-4abd-969b-ec5ebc937284 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:42.396381 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:42.432355 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941d06b-fba4-4abd-969b-ec5ebc937284 tempest-VolumeQuotaClassesTest-381321833 tempest-VolumeQuotaClassesTest-381321833-project-admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:42.486134 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:42.486759 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:42.487063 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:42.487336 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] PATH_INFO: `/v3/users/04a4d2bb164544509255bcb5b68929e8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:42.488452 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a4d2bb164544509255bcb5b68929e8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:42.490353 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:42.537616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a4d2bb164544509255bcb5b68929e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:42.538279 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a4d2bb164544509255bcb5b68929e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:42.549988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a4d2bb164544509255bcb5b68929e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:42.557161 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a4d2bb164544509255bcb5b68929e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:42.563966 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a4d2bb164544509255bcb5b68929e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:42.570414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1454/2884] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:42 2022] DELETE /identity/v3/users/04a4d2bb164544509255bcb5b68929e8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:42.574902 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058084c4-8006-468c-b875-571727a7d1bc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:42.575383 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058084c4-8006-468c-b875-571727a7d1bc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:42.575506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058084c4-8006-468c-b875-571727a7d1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:42.594337 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-058084c4-8006-468c-b875-571727a7d1bc None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:42.595562 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-058084c4-8006-468c-b875-571727a7d1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:42.641634 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:42.654858 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1431/2885] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:42.665011 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:42.666414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:42.724471 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a8a47-2def-4664-aaad-1bba57b24c15 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:42.729453 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577edb6-c5ca-4e40-8764-e503da0eb4c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:42.730054 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577edb6-c5ca-4e40-8764-e503da0eb4c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:42.730329 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577edb6-c5ca-4e40-8764-e503da0eb4c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:42.730620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577edb6-c5ca-4e40-8764-e503da0eb4c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:42.737597 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:42.740180 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577edb6-c5ca-4e40-8764-e503da0eb4c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:42.743102 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577edb6-c5ca-4e40-8764-e503da0eb4c5 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:42.753708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1455/2886] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:43.021755 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058084c4-8006-468c-b875-571727a7d1bc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:43.023708 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-058084c4-8006-468c-b875-571727a7d1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:43.041695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058084c4-8006-468c-b875-571727a7d1bc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:43.048129 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:43.049001 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:43.049314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:43.049589 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] PATH_INFO: `/v3/projects/163396fda144470692ee70659433dde2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:43.050768 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=163396fda144470692ee70659433dde2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:43.052713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:43.100187 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163396fda144470692ee70659433dde2', 'request_id': 'req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.106249 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163396fda144470692ee70659433dde2', 'request_id': 'req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.111866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163396fda144470692ee70659433dde2', 'request_id': 'req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.124988 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] The token cache is being invalidate because project 163396fda144470692ee70659433dde2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:43.125273 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.126040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.127541 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1432/2887] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:43 2022] DELETE /identity/v3/projects/163396fda144470692ee70659433dde2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:43.131676 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577edb6-c5ca-4e40-8764-e503da0eb4c5 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:43.141951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577edb6-c5ca-4e40-8764-e503da0eb4c5 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:43.198048 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1674a43-53c4-45cc-9d0b-70a193108807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:43.198667 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1674a43-53c4-45cc-9d0b-70a193108807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:43.198944 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1674a43-53c4-45cc-9d0b-70a193108807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:43.199245 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1674a43-53c4-45cc-9d0b-70a193108807 admin admin] PATH_INFO: `/v3/users/d7d675550088431dacebc11a337b7100` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:43.200332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1674a43-53c4-45cc-9d0b-70a193108807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d675550088431dacebc11a337b7100)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:43.202251 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1674a43-53c4-45cc-9d0b-70a193108807 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:43.246678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c1674a43-53c4-45cc-9d0b-70a193108807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d675550088431dacebc11a337b7100'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.247315 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c1674a43-53c4-45cc-9d0b-70a193108807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d675550088431dacebc11a337b7100'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.259383 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c1674a43-53c4-45cc-9d0b-70a193108807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d675550088431dacebc11a337b7100'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.266547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c1674a43-53c4-45cc-9d0b-70a193108807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d675550088431dacebc11a337b7100'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.273503 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c1674a43-53c4-45cc-9d0b-70a193108807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d675550088431dacebc11a337b7100'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.279218 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1456/2888] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:43 2022] DELETE /identity/v3/users/d7d675550088431dacebc11a337b7100 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:43.552553 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:43.554014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:43.567726 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741b11f-a6d4-4ea2-bd49-6a693d9757e9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:43.624392 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:43.625038 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:43.625198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:43.625471 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] PATH_INFO: `/v3/projects/556c774bedd942ce8f513b3cfa1b1be0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:43.626670 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=556c774bedd942ce8f513b3cfa1b1be0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:43.628594 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:43.675202 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '556c774bedd942ce8f513b3cfa1b1be0', 'request_id': 'req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.681426 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '556c774bedd942ce8f513b3cfa1b1be0', 'request_id': 'req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.687292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '556c774bedd942ce8f513b3cfa1b1be0', 'request_id': 'req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.700160 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] The token cache is being invalidate because project 556c774bedd942ce8f513b3cfa1b1be0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:43.700451 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.701079 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e74ca32-18c1-4ee9-a292-61c42a2fa2b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:43.702644 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1433/2889] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:43 2022] DELETE /identity/v3/projects/556c774bedd942ce8f513b3cfa1b1be0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:43.707575 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1cc58-c290-4f55-b0b0-b7e3fd0c0601 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:43.708326 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1cc58-c290-4f55-b0b0-b7e3fd0c0601 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:43.708454 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1cc58-c290-4f55-b0b0-b7e3fd0c0601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:43.728309 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3fb1cc58-c290-4f55-b0b0-b7e3fd0c0601 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:43.729507 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3fb1cc58-c290-4f55-b0b0-b7e3fd0c0601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:43.786366 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1457/2890] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:43.794135 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6c745-af81-454e-8834-99782d2b44cd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:43.794423 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6c745-af81-454e-8834-99782d2b44cd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:43.794703 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6c745-af81-454e-8834-99782d2b44cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:43.811508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:43.817764 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-82e6c745-af81-454e-8834-99782d2b44cd None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:43.819629 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-82e6c745-af81-454e-8834-99782d2b44cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:43.828581 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1434/2891] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:22:43.832861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1cc58-c290-4f55-b0b0-b7e3fd0c0601 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:43.842881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1cc58-c290-4f55-b0b0-b7e3fd0c0601 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:43.847460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46507d89-faff-47c2-aa53-446ab3b38d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:43.848056 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46507d89-faff-47c2-aa53-446ab3b38d97 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:43.848337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46507d89-faff-47c2-aa53-446ab3b38d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:43.848611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46507d89-faff-47c2-aa53-446ab3b38d97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:43.849253 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46507d89-faff-47c2-aa53-446ab3b38d97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:43.849554 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46507d89-faff-47c2-aa53-446ab3b38d97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:43.851179 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46507d89-faff-47c2-aa53-446ab3b38d97 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:43.862915 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1458/2892] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:43.868162 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6c745-af81-454e-8834-99782d2b44cd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:43.877344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6c745-af81-454e-8834-99782d2b44cd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:43.881818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b4192-d9d6-47b9-ac05-aa991edf4b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:43.883154 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b4192-d9d6-47b9-ac05-aa991edf4b58 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:43.883432 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b4192-d9d6-47b9-ac05-aa991edf4b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:43.883781 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b4192-d9d6-47b9-ac05-aa991edf4b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:43.884413 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b4192-d9d6-47b9-ac05-aa991edf4b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:43.886293 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b4192-d9d6-47b9-ac05-aa991edf4b58 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:43.912641 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1435/2893] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:43 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:43.916838 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46507d89-faff-47c2-aa53-446ab3b38d97 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:43.921987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:43.928072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46507d89-faff-47c2-aa53-446ab3b38d97 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:43.985149 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337f4a2-0287-4a22-944d-c8ab90ae1a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:43.985746 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f4a2-0287-4a22-944d-c8ab90ae1a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:43.986024 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f4a2-0287-4a22-944d-c8ab90ae1a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:43.986296 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f4a2-0287-4a22-944d-c8ab90ae1a23 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:43.986914 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337f4a2-0287-4a22-944d-c8ab90ae1a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:43.988843 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337f4a2-0287-4a22-944d-c8ab90ae1a23 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:44.010703 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1459/2894] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:43 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:44.014779 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b4192-d9d6-47b9-ac05-aa991edf4b58 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:44.025287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b4192-d9d6-47b9-ac05-aa991edf4b58 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:44.029829 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33999608-370f-4eec-91e4-88485ce5fb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:44.030417 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999608-370f-4eec-91e4-88485ce5fb61 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:44.030693 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999608-370f-4eec-91e4-88485ce5fb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:44.030960 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999608-370f-4eec-91e4-88485ce5fb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:44.031647 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33999608-370f-4eec-91e4-88485ce5fb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:44.033222 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33999608-370f-4eec-91e4-88485ce5fb61 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:44.043109 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1436/2895] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:22:44.047089 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337f4a2-0287-4a22-944d-c8ab90ae1a23 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:44.057387 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337f4a2-0287-4a22-944d-c8ab90ae1a23 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:44.061896 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cb759-5deb-4a04-95c7-a1957a4677bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:44.062476 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cb759-5deb-4a04-95c7-a1957a4677bb admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:44.062753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cb759-5deb-4a04-95c7-a1957a4677bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:44.063050 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cb759-5deb-4a04-95c7-a1957a4677bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:44.063680 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7cb759-5deb-4a04-95c7-a1957a4677bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:44.065281 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7cb759-5deb-4a04-95c7-a1957a4677bb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:44.076264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1460/2896] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:44.080658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33999608-370f-4eec-91e4-88485ce5fb61 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:44.090705 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33999608-370f-4eec-91e4-88485ce5fb61 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:44.095271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d221cc-dcd5-4cd8-9df7-8c01d1b6c1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:44.095906 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d221cc-dcd5-4cd8-9df7-8c01d1b6c1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:44.096183 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d221cc-dcd5-4cd8-9df7-8c01d1b6c1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:44.096453 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d221cc-dcd5-4cd8-9df7-8c01d1b6c1e9 admin admin] PATH_INFO: `/v3/projects/1041adcaaaf945d8827762ffbace6629/users/5c980aa6750b436b85adbc1507afc653/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:44.111067 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d221cc-dcd5-4cd8-9df7-8c01d1b6c1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1041adcaaaf945d8827762ffbace6629, user_id=5c980aa6750b436b85adbc1507afc653, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:44.113681 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d221cc-dcd5-4cd8-9df7-8c01d1b6c1e9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:44.127075 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1437/2897] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:44 2022] PUT /identity/v3/projects/1041adcaaaf945d8827762ffbace6629/users/5c980aa6750b436b85adbc1507afc653/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:44.138550 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8544b9-ad41-493a-955a-e8dc0ae7aab1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:44.138782 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8544b9-ad41-493a-955a-e8dc0ae7aab1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:44.139093 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8544b9-ad41-493a-955a-e8dc0ae7aab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:44.169377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4e8544b9-ad41-493a-955a-e8dc0ae7aab1 None None] MFA Rules not processed for user `5c980aa6750b436b85adbc1507afc653`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:44.170612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4e8544b9-ad41-493a-955a-e8dc0ae7aab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:44.237984 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1461/2898] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:44.250713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d221cc-dcd5-4cd8-9df7-8c01d1b6c1e9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:44.252397 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-02d221cc-dcd5-4cd8-9df7-8c01d1b6c1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:44.310739 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d221cc-dcd5-4cd8-9df7-8c01d1b6c1e9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:44.314416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529292-d255-4fbb-b68c-46337be405d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:44.315040 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529292-d255-4fbb-b68c-46337be405d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:44.315340 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529292-d255-4fbb-b68c-46337be405d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:44.315609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529292-d255-4fbb-b68c-46337be405d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:44.323645 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4529292-d255-4fbb-b68c-46337be405d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:44.325039 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4529292-d255-4fbb-b68c-46337be405d4 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:44.333964 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1438/2899] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:44.353338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8544b9-ad41-493a-955a-e8dc0ae7aab1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:44.417885 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8544b9-ad41-493a-955a-e8dc0ae7aab1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:44.422488 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:44.423152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:44.423428 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:44.423700 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] PATH_INFO: `/v3/users/5c980aa6750b436b85adbc1507afc653` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:44.424839 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c980aa6750b436b85adbc1507afc653)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:44.426738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:44.439710 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:44.473409 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c980aa6750b436b85adbc1507afc653'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:44.474074 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c980aa6750b436b85adbc1507afc653'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:44.486449 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c980aa6750b436b85adbc1507afc653'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:44.494531 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c980aa6750b436b85adbc1507afc653'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:44.501306 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c980aa6750b436b85adbc1507afc653'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:44.507266 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1462/2900] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:44 2022] DELETE /identity/v3/users/5c980aa6750b436b85adbc1507afc653 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:44.511862 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67049f1-1576-4481-9519-79e5364bc318 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:44.512147 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67049f1-1576-4481-9519-79e5364bc318 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:44.512421 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67049f1-1576-4481-9519-79e5364bc318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:44.534523 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d67049f1-1576-4481-9519-79e5364bc318 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:44.535762 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d67049f1-1576-4481-9519-79e5364bc318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:44.594243 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1439/2901] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:44.604663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:44.606088 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:44.662275 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057abf10-bbf7-4d22-8c30-65e0718c9eb8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:44.665875 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceef48e-2aca-4c75-b53e-b9f21b34c6c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:44.666468 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceef48e-2aca-4c75-b53e-b9f21b34c6c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:44.666743 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceef48e-2aca-4c75-b53e-b9f21b34c6c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:44.667031 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceef48e-2aca-4c75-b53e-b9f21b34c6c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:44.676032 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceef48e-2aca-4c75-b53e-b9f21b34c6c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:44.677389 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceef48e-2aca-4c75-b53e-b9f21b34c6c4 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:44.687436 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1463/2902] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:44.951146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67049f1-1576-4481-9519-79e5364bc318 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:44.952605 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d67049f1-1576-4481-9519-79e5364bc318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:44.965199 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67049f1-1576-4481-9519-79e5364bc318 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:44.969800 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:44.970431 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:44.970866 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:44.971178 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] PATH_INFO: `/v3/projects/1041adcaaaf945d8827762ffbace6629` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:44.972292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1041adcaaaf945d8827762ffbace6629)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:44.974277 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.025446 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:45.032727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1041adcaaaf945d8827762ffbace6629', 'request_id': 'req-38eb45fc-ac41-450a-a294-f2d0582db6e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:45.039797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1041adcaaaf945d8827762ffbace6629', 'request_id': 'req-38eb45fc-ac41-450a-a294-f2d0582db6e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:45.045198 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1041adcaaaf945d8827762ffbace6629', 'request_id': 'req-38eb45fc-ac41-450a-a294-f2d0582db6e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:45.057295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] The token cache is being invalidate because project 1041adcaaaf945d8827762ffbace6629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:45.057583 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:45.058196 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-38eb45fc-ac41-450a-a294-f2d0582db6e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:45.059750 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1440/2903] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:44 2022] DELETE /identity/v3/projects/1041adcaaaf945d8827762ffbace6629 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:45.064849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fcc84-121b-4c7a-bbbe-588780573aac None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.065360 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fcc84-121b-4c7a-bbbe-588780573aac None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.065460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fcc84-121b-4c7a-bbbe-588780573aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.084760 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ea6fcc84-121b-4c7a-bbbe-588780573aac None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:45.085951 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ea6fcc84-121b-4c7a-bbbe-588780573aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:45.142920 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1464/2904] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:45.150752 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d7f11-0f78-469e-b104-eb633f96a8fc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.151080 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d7f11-0f78-469e-b104-eb633f96a8fc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.151359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d7f11-0f78-469e-b104-eb633f96a8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.171117 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-509d7f11-0f78-469e-b104-eb633f96a8fc None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:45.172304 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-509d7f11-0f78-469e-b104-eb633f96a8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:45.180736 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1441/2905] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:45.184930 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fcc84-121b-4c7a-bbbe-588780573aac None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.195113 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fcc84-121b-4c7a-bbbe-588780573aac None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:45.199553 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08289d8-493b-41b1-8b67-7902b0597204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:45.200140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08289d8-493b-41b1-8b67-7902b0597204 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.200414 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08289d8-493b-41b1-8b67-7902b0597204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.200708 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08289d8-493b-41b1-8b67-7902b0597204 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.201335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08289d8-493b-41b1-8b67-7902b0597204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:45.201630 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08289d8-493b-41b1-8b67-7902b0597204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:45.203212 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08289d8-493b-41b1-8b67-7902b0597204 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.214738 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1465/2906] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:45.219146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d7f11-0f78-469e-b104-eb633f96a8fc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.229778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d7f11-0f78-469e-b104-eb633f96a8fc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:45.234365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a704ee-e343-4db3-893c-3f0743f90002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:45.234955 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a704ee-e343-4db3-893c-3f0743f90002 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.235265 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a704ee-e343-4db3-893c-3f0743f90002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.235531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a704ee-e343-4db3-893c-3f0743f90002 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.236149 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a704ee-e343-4db3-893c-3f0743f90002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:45.238068 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a704ee-e343-4db3-893c-3f0743f90002 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.265244 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1442/2907] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:45 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:45.269463 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08289d8-493b-41b1-8b67-7902b0597204 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.279970 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08289d8-493b-41b1-8b67-7902b0597204 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:45.296923 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:45.337216 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb5834-8a08-484d-8b2a-b5622c326d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:45.337813 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb5834-8a08-484d-8b2a-b5622c326d6a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.338092 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb5834-8a08-484d-8b2a-b5622c326d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.338363 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb5834-8a08-484d-8b2a-b5622c326d6a admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.338988 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb5834-8a08-484d-8b2a-b5622c326d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:45.340906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb5834-8a08-484d-8b2a-b5622c326d6a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.361682 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1466/2908] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:45 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:45.365818 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a704ee-e343-4db3-893c-3f0743f90002 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.376390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a704ee-e343-4db3-893c-3f0743f90002 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:45.381114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc215582-e166-4ba2-841d-9e2bcce7d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:45.381700 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc215582-e166-4ba2-841d-9e2bcce7d917 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.381976 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc215582-e166-4ba2-841d-9e2bcce7d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.382246 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc215582-e166-4ba2-841d-9e2bcce7d917 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.382873 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc215582-e166-4ba2-841d-9e2bcce7d917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:45.384491 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc215582-e166-4ba2-841d-9e2bcce7d917 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.394354 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1443/2909] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:45.398670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb5834-8a08-484d-8b2a-b5622c326d6a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.408761 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb5834-8a08-484d-8b2a-b5622c326d6a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:45.414005 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61441394-c7b3-4462-8e53-b0f56a00393a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:45.414612 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441394-c7b3-4462-8e53-b0f56a00393a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.414897 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441394-c7b3-4462-8e53-b0f56a00393a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.415235 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441394-c7b3-4462-8e53-b0f56a00393a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.415868 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61441394-c7b3-4462-8e53-b0f56a00393a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:45.417442 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61441394-c7b3-4462-8e53-b0f56a00393a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.428072 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1467/2910] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:22:45.432185 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc215582-e166-4ba2-841d-9e2bcce7d917 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.442613 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc215582-e166-4ba2-841d-9e2bcce7d917 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:45.447367 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff453a4-75bc-4613-ae32-627461491b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:45.447962 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff453a4-75bc-4613-ae32-627461491b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.448239 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff453a4-75bc-4613-ae32-627461491b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.448506 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff453a4-75bc-4613-ae32-627461491b5a admin admin] PATH_INFO: `/v3/projects/e29b5cfe7c6d4cc4828d2a88244b81a0/users/82e424337c3b4b638c4e67ca10f35127/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.465727 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff453a4-75bc-4613-ae32-627461491b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29b5cfe7c6d4cc4828d2a88244b81a0, user_id=82e424337c3b4b638c4e67ca10f35127, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:45.468393 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff453a4-75bc-4613-ae32-627461491b5a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.481891 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1444/2911] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:45 2022] PUT /identity/v3/projects/e29b5cfe7c6d4cc4828d2a88244b81a0/users/82e424337c3b4b638c4e67ca10f35127/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:45.492081 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978eff48-e34b-413d-99c6-e78665114460 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.492457 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978eff48-e34b-413d-99c6-e78665114460 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.492799 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978eff48-e34b-413d-99c6-e78665114460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.520421 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-978eff48-e34b-413d-99c6-e78665114460 None None] MFA Rules not processed for user `82e424337c3b4b638c4e67ca10f35127`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:45.521877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-978eff48-e34b-413d-99c6-e78665114460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:45.590351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1468/2912] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:45.594967 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff453a4-75bc-4613-ae32-627461491b5a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.655406 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff453a4-75bc-4613-ae32-627461491b5a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:45.660218 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd79065-b185-4feb-9825-6ba7a1c133b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:45.660830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79065-b185-4feb-9825-6ba7a1c133b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.661114 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79065-b185-4feb-9825-6ba7a1c133b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.661390 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79065-b185-4feb-9825-6ba7a1c133b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.662025 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd79065-b185-4feb-9825-6ba7a1c133b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:45.664466 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd79065-b185-4feb-9825-6ba7a1c133b7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.691508 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1445/2913] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:45 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:45.696100 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978eff48-e34b-413d-99c6-e78665114460 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.706964 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978eff48-e34b-413d-99c6-e78665114460 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:45.763330 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd08c8-7264-4382-9861-15928ae0dffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:45.763933 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd08c8-7264-4382-9861-15928ae0dffc admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.764236 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd08c8-7264-4382-9861-15928ae0dffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.764505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd08c8-7264-4382-9861-15928ae0dffc admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.765151 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd08c8-7264-4382-9861-15928ae0dffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:45.767130 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd08c8-7264-4382-9861-15928ae0dffc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.788191 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1469/2914] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:45 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:45.792175 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd79065-b185-4feb-9825-6ba7a1c133b7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.802256 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd79065-b185-4feb-9825-6ba7a1c133b7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:45.806904 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b6ff2-8b82-453e-a25d-ca65dd144e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:45.807536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6ff2-8b82-453e-a25d-ca65dd144e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.807814 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6ff2-8b82-453e-a25d-ca65dd144e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.808086 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6ff2-8b82-453e-a25d-ca65dd144e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.808717 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b6ff2-8b82-453e-a25d-ca65dd144e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:45.810292 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b6ff2-8b82-453e-a25d-ca65dd144e54 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.822125 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1446/2915] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:45.826268 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd08c8-7264-4382-9861-15928ae0dffc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.840935 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd08c8-7264-4382-9861-15928ae0dffc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:45.848129 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2d549-6637-4160-a742-c7c10c25a5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:45.849114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2d549-6637-4160-a742-c7c10c25a5ae admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.849570 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2d549-6637-4160-a742-c7c10c25a5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.849993 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2d549-6637-4160-a742-c7c10c25a5ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.850663 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2d549-6637-4160-a742-c7c10c25a5ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:45.852380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2d549-6637-4160-a742-c7c10c25a5ae admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.862798 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1470/2916] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:22:45.867780 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b6ff2-8b82-453e-a25d-ca65dd144e54 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:45.878248 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b6ff2-8b82-453e-a25d-ca65dd144e54 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:45.883637 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004bf9d-acb8-4e49-8d50-6f5843483f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:45.884295 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004bf9d-acb8-4e49-8d50-6f5843483f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.884600 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004bf9d-acb8-4e49-8d50-6f5843483f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.884875 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004bf9d-acb8-4e49-8d50-6f5843483f45 admin admin] PATH_INFO: `/v3/projects/8817f344f7bd44e59bb990fc8667090b/users/9a7a32461eeb4a2590ce7a91a8bafd92/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.901369 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004bf9d-acb8-4e49-8d50-6f5843483f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8817f344f7bd44e59bb990fc8667090b, user_id=9a7a32461eeb4a2590ce7a91a8bafd92, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:45.904073 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004bf9d-acb8-4e49-8d50-6f5843483f45 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:45.916653 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1447/2917] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:45 2022] PUT /identity/v3/projects/8817f344f7bd44e59bb990fc8667090b/users/9a7a32461eeb4a2590ce7a91a8bafd92/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:22:45.928109 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf990da-5206-4f9e-a45a-43a1198c3e66 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:45.928397 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf990da-5206-4f9e-a45a-43a1198c3e66 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:45.928669 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf990da-5206-4f9e-a45a-43a1198c3e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:45.942366 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:45.957808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1cf990da-5206-4f9e-a45a-43a1198c3e66 None None] MFA Rules not processed for user `9a7a32461eeb4a2590ce7a91a8bafd92`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:45.959284 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1cf990da-5206-4f9e-a45a-43a1198c3e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:46.023500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1471/2918] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:46.027967 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004bf9d-acb8-4e49-8d50-6f5843483f45 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:46.045032 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:46.091308 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004bf9d-acb8-4e49-8d50-6f5843483f45 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:46.096171 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502d434-e3b7-4545-9ad1-2f4e26714b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:46.096766 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502d434-e3b7-4545-9ad1-2f4e26714b3d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.097140 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502d434-e3b7-4545-9ad1-2f4e26714b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.097314 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502d434-e3b7-4545-9ad1-2f4e26714b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.097926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502d434-e3b7-4545-9ad1-2f4e26714b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:46.099825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502d434-e3b7-4545-9ad1-2f4e26714b3d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:46.126450 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1448/2919] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:46 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:46.130817 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf990da-5206-4f9e-a45a-43a1198c3e66 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:46.142371 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf990da-5206-4f9e-a45a-43a1198c3e66 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:46.202808 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6547d54-544f-460b-91a4-2e27a95dca38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:46.203592 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6547d54-544f-460b-91a4-2e27a95dca38 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.203869 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6547d54-544f-460b-91a4-2e27a95dca38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.204140 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6547d54-544f-460b-91a4-2e27a95dca38 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.204916 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6547d54-544f-460b-91a4-2e27a95dca38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:46.207033 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6547d54-544f-460b-91a4-2e27a95dca38 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:46.231751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1472/2920] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:46 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:22:46.236444 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502d434-e3b7-4545-9ad1-2f4e26714b3d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:46.247959 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502d434-e3b7-4545-9ad1-2f4e26714b3d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:46.252524 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9106a-682e-4376-acb0-a58de11b314e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:46.253116 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9106a-682e-4376-acb0-a58de11b314e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.253400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9106a-682e-4376-acb0-a58de11b314e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.253670 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9106a-682e-4376-acb0-a58de11b314e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.254332 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9106a-682e-4376-acb0-a58de11b314e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:46.255947 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9106a-682e-4376-acb0-a58de11b314e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:46.266828 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1449/2921] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:46.270936 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6547d54-544f-460b-91a4-2e27a95dca38 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:46.280921 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6547d54-544f-460b-91a4-2e27a95dca38 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:46.285563 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc608fe9-71bc-45bc-9234-8d81f33bcf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:46.286207 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608fe9-71bc-45bc-9234-8d81f33bcf94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.286483 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608fe9-71bc-45bc-9234-8d81f33bcf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.286751 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608fe9-71bc-45bc-9234-8d81f33bcf94 admin admin] PATH_INFO: `/v3/projects/37918c6ec8974954a547386ca12de3fe/users/abfc0757c59c4d888e58624a2499d1de/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.301177 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc608fe9-71bc-45bc-9234-8d81f33bcf94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37918c6ec8974954a547386ca12de3fe, user_id=abfc0757c59c4d888e58624a2499d1de, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:46.304324 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc608fe9-71bc-45bc-9234-8d81f33bcf94 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:46.316925 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1473/2922] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:46 2022] PUT /identity/v3/projects/37918c6ec8974954a547386ca12de3fe/users/abfc0757c59c4d888e58624a2499d1de/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:46.321536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9106a-682e-4376-acb0-a58de11b314e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:46.331639 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9106a-682e-4376-acb0-a58de11b314e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:46.386597 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b821f7-6f77-4ac9-9674-e7e897b2d754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:46.387223 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b821f7-6f77-4ac9-9674-e7e897b2d754 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.387504 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b821f7-6f77-4ac9-9674-e7e897b2d754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.387792 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b821f7-6f77-4ac9-9674-e7e897b2d754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.388427 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b821f7-6f77-4ac9-9674-e7e897b2d754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:46.389991 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b821f7-6f77-4ac9-9674-e7e897b2d754 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:46.400083 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1450/2923] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:46.404181 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc608fe9-71bc-45bc-9234-8d81f33bcf94 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:46.415152 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc608fe9-71bc-45bc-9234-8d81f33bcf94 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:46.419750 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7067b-d6cd-4eea-8d9d-87e63ea2a70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:46.420402 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7067b-d6cd-4eea-8d9d-87e63ea2a70a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.420681 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7067b-d6cd-4eea-8d9d-87e63ea2a70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.420955 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7067b-d6cd-4eea-8d9d-87e63ea2a70a admin admin] PATH_INFO: `/v3/projects/37918c6ec8974954a547386ca12de3fe/users/abfc0757c59c4d888e58624a2499d1de/roles/470667397ce74e4dbf0b4605ade12c23` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.422906 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7067b-d6cd-4eea-8d9d-87e63ea2a70a admin admin] RBAC: Authorizing `identity:create_grant(project_id=37918c6ec8974954a547386ca12de3fe, user_id=abfc0757c59c4d888e58624a2499d1de, role_id=470667397ce74e4dbf0b4605ade12c23)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:46.425589 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7067b-d6cd-4eea-8d9d-87e63ea2a70a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:46.436477 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1474/2924] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:46 2022] PUT /identity/v3/projects/37918c6ec8974954a547386ca12de3fe/users/abfc0757c59c4d888e58624a2499d1de/roles/470667397ce74e4dbf0b4605ade12c23 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:46.446535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f6de6-20f8-4a04-8543-41007cc3e144 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.446825 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f6de6-20f8-4a04-8543-41007cc3e144 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.447164 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f6de6-20f8-4a04-8543-41007cc3e144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.474859 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-944f6de6-20f8-4a04-8543-41007cc3e144 None None] MFA Rules not processed for user `abfc0757c59c4d888e58624a2499d1de`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:46.476609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-944f6de6-20f8-4a04-8543-41007cc3e144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:46.544535 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1451/2925] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:46.555928 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7067b-d6cd-4eea-8d9d-87e63ea2a70a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:46.557678 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c3c7067b-d6cd-4eea-8d9d-87e63ea2a70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:46.613271 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:46.617949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7067b-d6cd-4eea-8d9d-87e63ea2a70a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:46.622125 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ac15f-ea65-431e-848a-b737ea870de0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:46.622739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ac15f-ea65-431e-848a-b737ea870de0 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:46.623044 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ac15f-ea65-431e-848a-b737ea870de0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:46.623319 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ac15f-ea65-431e-848a-b737ea870de0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:46.678825 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ac15f-ea65-431e-848a-b737ea870de0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:46.680289 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ac15f-ea65-431e-848a-b737ea870de0 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:46.689278 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1475/2926] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:48.369797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f6de6-20f8-4a04-8543-41007cc3e144 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:48.371219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-944f6de6-20f8-4a04-8543-41007cc3e144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:48.382961 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f6de6-20f8-4a04-8543-41007cc3e144 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:48.386457 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3de84-e0f6-498e-aa7f-044d19bae05c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:48.387078 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3de84-e0f6-498e-aa7f-044d19bae05c service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:48.387360 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3de84-e0f6-498e-aa7f-044d19bae05c service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:48.387634 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3de84-e0f6-498e-aa7f-044d19bae05c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:48.395713 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:48.450960 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3de84-e0f6-498e-aa7f-044d19bae05c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:48.452490 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3de84-e0f6-498e-aa7f-044d19bae05c service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:48.461712 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1452/2927] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:48.992864 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ac15f-ea65-431e-848a-b737ea870de0 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:49.001776 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ac15f-ea65-431e-848a-b737ea870de0 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:49.005447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17363a-8d93-46dd-9729-7ffbc37eaaa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3030e2c49614aee892c7693da6b73bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:49.006038 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17363a-8d93-46dd-9729-7ffbc37eaaa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:49.006338 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17363a-8d93-46dd-9729-7ffbc37eaaa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:49.006608 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17363a-8d93-46dd-9729-7ffbc37eaaa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:49.016193 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17363a-8d93-46dd-9729-7ffbc37eaaa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:49.017598 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17363a-8d93-46dd-9729-7ffbc37eaaa6 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:49.027440 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1476/2928] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:52.881328 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3de84-e0f6-498e-aa7f-044d19bae05c service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:52.943601 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3de84-e0f6-498e-aa7f-044d19bae05c service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:52.948702 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:52.949320 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:52.949607 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:52.949883 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] PATH_INFO: `/v3/users/82e424337c3b4b638c4e67ca10f35127` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:52.951016 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82e424337c3b4b638c4e67ca10f35127)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:52.952924 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:52.999782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82e424337c3b4b638c4e67ca10f35127'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.012261 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82e424337c3b4b638c4e67ca10f35127'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.012926 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82e424337c3b4b638c4e67ca10f35127'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.013526 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82e424337c3b4b638c4e67ca10f35127'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.021363 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82e424337c3b4b638c4e67ca10f35127'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.034146 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1453/2929] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:52 2022] DELETE /identity/v3/users/82e424337c3b4b638c4e67ca10f35127 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:53.038702 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945c031-f532-48fe-9b59-9a4ac23de0de None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.039198 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945c031-f532-48fe-9b59-9a4ac23de0de None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.039332 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945c031-f532-48fe-9b59-9a4ac23de0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.058244 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5945c031-f532-48fe-9b59-9a4ac23de0de None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:53.059460 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5945c031-f532-48fe-9b59-9a4ac23de0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:53.115637 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1477/2930] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:53.125830 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:53.127300 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:53.158378 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:53.186566 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02e3bd-7539-43cf-b725-181e3ba0ccc2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:53.190182 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de692c3-49fd-41b4-910a-eefde91f842f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:53.190774 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de692c3-49fd-41b4-910a-eefde91f842f service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.191077 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de692c3-49fd-41b4-910a-eefde91f842f service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.191346 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de692c3-49fd-41b4-910a-eefde91f842f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.200130 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de692c3-49fd-41b4-910a-eefde91f842f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:53.201566 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de692c3-49fd-41b4-910a-eefde91f842f service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:53.211436 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1454/2931] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:53.481506 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945c031-f532-48fe-9b59-9a4ac23de0de None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:53.482926 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5945c031-f532-48fe-9b59-9a4ac23de0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:53.495504 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945c031-f532-48fe-9b59-9a4ac23de0de None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:53.500189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:53.500821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.501222 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.501513 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] PATH_INFO: `/v3/projects/e29b5cfe7c6d4cc4828d2a88244b81a0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.502616 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e29b5cfe7c6d4cc4828d2a88244b81a0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:53.504543 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:53.552608 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e29b5cfe7c6d4cc4828d2a88244b81a0', 'request_id': 'req-bb3965b4-ec16-41bb-bced-96715a05bf41'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.560416 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e29b5cfe7c6d4cc4828d2a88244b81a0', 'request_id': 'req-bb3965b4-ec16-41bb-bced-96715a05bf41'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.566724 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e29b5cfe7c6d4cc4828d2a88244b81a0', 'request_id': 'req-bb3965b4-ec16-41bb-bced-96715a05bf41'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.576884 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] The token cache is being invalidate because project e29b5cfe7c6d4cc4828d2a88244b81a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:53.577170 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.577829 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.579426 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1478/2932] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:53 2022] DELETE /identity/v3/projects/e29b5cfe7c6d4cc4828d2a88244b81a0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:53.583649 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de692c3-49fd-41b4-910a-eefde91f842f service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:53.593795 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de692c3-49fd-41b4-910a-eefde91f842f service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:53.651127 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:53.651672 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:53.652014 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:53.652285 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] PATH_INFO: `/v3/users/9a7a32461eeb4a2590ce7a91a8bafd92` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:53.653400 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a7a32461eeb4a2590ce7a91a8bafd92)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:53.655323 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:53.702513 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a7a32461eeb4a2590ce7a91a8bafd92'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.714914 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a7a32461eeb4a2590ce7a91a8bafd92'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.715609 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a7a32461eeb4a2590ce7a91a8bafd92'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.716214 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a7a32461eeb4a2590ce7a91a8bafd92'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.723569 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a7a32461eeb4a2590ce7a91a8bafd92'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:53.737227 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1455/2933] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:53 2022] DELETE /identity/v3/users/9a7a32461eeb4a2590ce7a91a8bafd92 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:54.003877 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:54.005347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:54.018611 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3965b4-ec16-41bb-bced-96715a05bf41 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:54.028335 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:54.078640 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:54.079377 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:54.079580 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:54.079849 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] PATH_INFO: `/v3/projects/8817f344f7bd44e59bb990fc8667090b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:54.080992 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8817f344f7bd44e59bb990fc8667090b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:54.082889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:54.130141 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8817f344f7bd44e59bb990fc8667090b', 'request_id': 'req-cc37826e-52a6-4f05-8a57-eb9e474a2063'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.137264 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8817f344f7bd44e59bb990fc8667090b', 'request_id': 'req-cc37826e-52a6-4f05-8a57-eb9e474a2063'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.143485 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8817f344f7bd44e59bb990fc8667090b', 'request_id': 'req-cc37826e-52a6-4f05-8a57-eb9e474a2063'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.153547 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] The token cache is being invalidate because project 8817f344f7bd44e59bb990fc8667090b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:54.153832 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.154459 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.155949 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1479/2934] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:54 2022] DELETE /identity/v3/projects/8817f344f7bd44e59bb990fc8667090b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:54.159943 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:54.161372 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:54.174550 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af244f29-c58a-42c8-9df2-8baa1e54f821 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:54.220365 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:54.233350 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01bf54-4aaf-44e4-97d4-eb795bbb7108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:54.233978 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01bf54-4aaf-44e4-97d4-eb795bbb7108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:54.234262 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01bf54-4aaf-44e4-97d4-eb795bbb7108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:54.234538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01bf54-4aaf-44e4-97d4-eb795bbb7108 admin admin] PATH_INFO: `/v3/users/abfc0757c59c4d888e58624a2499d1de` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:54.235634 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01bf54-4aaf-44e4-97d4-eb795bbb7108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abfc0757c59c4d888e58624a2499d1de)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:54.237542 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01bf54-4aaf-44e4-97d4-eb795bbb7108 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:54.283725 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4a01bf54-4aaf-44e4-97d4-eb795bbb7108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abfc0757c59c4d888e58624a2499d1de'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.295721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4a01bf54-4aaf-44e4-97d4-eb795bbb7108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abfc0757c59c4d888e58624a2499d1de'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.296377 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4a01bf54-4aaf-44e4-97d4-eb795bbb7108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abfc0757c59c4d888e58624a2499d1de'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.296967 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4a01bf54-4aaf-44e4-97d4-eb795bbb7108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abfc0757c59c4d888e58624a2499d1de'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.304058 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4a01bf54-4aaf-44e4-97d4-eb795bbb7108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abfc0757c59c4d888e58624a2499d1de'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.315989 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1456/2935] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:22:54 2022] DELETE /identity/v3/users/abfc0757c59c4d888e58624a2499d1de => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:54.599725 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:54.601704 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:54.616049 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37826e-52a6-4f05-8a57-eb9e474a2063 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:54.674610 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:54.675337 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:54.675619 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:54.675895 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] PATH_INFO: `/v3/projects/37918c6ec8974954a547386ca12de3fe` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:54.677118 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=37918c6ec8974954a547386ca12de3fe)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:54.679183 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:54.729861 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:54.733395 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37918c6ec8974954a547386ca12de3fe', 'request_id': 'req-84692b31-933c-40d2-868a-185cec9631fa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.742505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37918c6ec8974954a547386ca12de3fe', 'request_id': 'req-84692b31-933c-40d2-868a-185cec9631fa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.749653 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37918c6ec8974954a547386ca12de3fe', 'request_id': 'req-84692b31-933c-40d2-868a-185cec9631fa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.760905 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] The token cache is being invalidate because project 37918c6ec8974954a547386ca12de3fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:22:54.761199 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.761866 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-84692b31-933c-40d2-868a-185cec9631fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:22:54.763380 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1480/2936] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:22:54 2022] DELETE /identity/v3/projects/37918c6ec8974954a547386ca12de3fe => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:54.768339 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b22a-8f90-4ebb-b55d-8968e6ca714c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:54.768626 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b22a-8f90-4ebb-b55d-8968e6ca714c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:54.768905 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b22a-8f90-4ebb-b55d-8968e6ca714c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:54.788658 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c689b22a-8f90-4ebb-b55d-8968e6ca714c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:54.789868 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c689b22a-8f90-4ebb-b55d-8968e6ca714c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:54.849207 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1457/2937] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:54.858351 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f178ab-728f-409a-aa59-cf99ce54bb6c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:54.858739 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f178ab-728f-409a-aa59-cf99ce54bb6c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:54.859090 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f178ab-728f-409a-aa59-cf99ce54bb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:54.879462 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f1f178ab-728f-409a-aa59-cf99ce54bb6c None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:54.880670 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f1f178ab-728f-409a-aa59-cf99ce54bb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:54.889295 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1481/2938] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:54.893538 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b22a-8f90-4ebb-b55d-8968e6ca714c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:54.903886 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b22a-8f90-4ebb-b55d-8968e6ca714c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:54.908445 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cc25b-9ce0-42d2-be51-2712b5f6ef18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:54.909031 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cc25b-9ce0-42d2-be51-2712b5f6ef18 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:54.909305 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cc25b-9ce0-42d2-be51-2712b5f6ef18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:54.909573 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cc25b-9ce0-42d2-be51-2712b5f6ef18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:54.910212 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cc25b-9ce0-42d2-be51-2712b5f6ef18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:22:54.910509 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cc25b-9ce0-42d2-be51-2712b5f6ef18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:54.912103 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cc25b-9ce0-42d2-be51-2712b5f6ef18 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:54.923161 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1458/2939] 10.222.0.86 () {58 vars in 1153 bytes} [Mon Aug 29 19:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:22:54.927448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f178ab-728f-409a-aa59-cf99ce54bb6c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:54.936650 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f178ab-728f-409a-aa59-cf99ce54bb6c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:54.941500 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06514fe9-706a-4a92-9e6e-0744321de42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:54.942122 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06514fe9-706a-4a92-9e6e-0744321de42b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:54.942400 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06514fe9-706a-4a92-9e6e-0744321de42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:54.942692 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06514fe9-706a-4a92-9e6e-0744321de42b admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:54.943347 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06514fe9-706a-4a92-9e6e-0744321de42b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:54.945809 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06514fe9-706a-4a92-9e6e-0744321de42b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:54.972447 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1482/2940] 10.222.0.86 () {60 vars in 1153 bytes} [Mon Aug 29 19:22:54 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:22:54.976662 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cc25b-9ce0-42d2-be51-2712b5f6ef18 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:54.986631 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cc25b-9ce0-42d2-be51-2712b5f6ef18 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:55.040118 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:22:55.048560 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c0952-fc65-43f4-88cf-a3f0191c52a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:55.049194 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c0952-fc65-43f4-88cf-a3f0191c52a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:55.049719 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c0952-fc65-43f4-88cf-a3f0191c52a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:55.050219 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c0952-fc65-43f4-88cf-a3f0191c52a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:55.051576 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c0952-fc65-43f4-88cf-a3f0191c52a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:55.054271 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c0952-fc65-43f4-88cf-a3f0191c52a6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:55.077028 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1459/2941] 10.222.0.86 () {60 vars in 1144 bytes} [Mon Aug 29 19:22:54 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:22:55.081239 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06514fe9-706a-4a92-9e6e-0744321de42b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:55.091747 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06514fe9-706a-4a92-9e6e-0744321de42b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:55.096265 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901bdc2-fe37-4bd1-b663-7ceaf535e018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:55.096856 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901bdc2-fe37-4bd1-b663-7ceaf535e018 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:55.097136 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901bdc2-fe37-4bd1-b663-7ceaf535e018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:55.097408 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901bdc2-fe37-4bd1-b663-7ceaf535e018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:55.098036 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901bdc2-fe37-4bd1-b663-7ceaf535e018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:55.099655 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901bdc2-fe37-4bd1-b663-7ceaf535e018 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:55.109777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1483/2942] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:55.113778 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c0952-fc65-43f4-88cf-a3f0191c52a6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:55.124154 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c0952-fc65-43f4-88cf-a3f0191c52a6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:55.128901 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a138e-b9ea-4c30-a051-908f0acfe6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:55.129487 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a138e-b9ea-4c30-a051-908f0acfe6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:55.129782 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a138e-b9ea-4c30-a051-908f0acfe6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:55.130042 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a138e-b9ea-4c30-a051-908f0acfe6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:55.130674 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a138e-b9ea-4c30-a051-908f0acfe6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:55.132945 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a138e-b9ea-4c30-a051-908f0acfe6a3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:55.143503 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1460/2943] 10.222.0.86 () {58 vars in 1122 bytes} [Mon Aug 29 19:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:22:55.147587 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901bdc2-fe37-4bd1-b663-7ceaf535e018 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:55.157565 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901bdc2-fe37-4bd1-b663-7ceaf535e018 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:55.162221 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f9a99-801b-4ec1-8954-3ba1b79a94b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:55.162860 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f9a99-801b-4ec1-8954-3ba1b79a94b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:55.163242 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f9a99-801b-4ec1-8954-3ba1b79a94b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:55.163516 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f9a99-801b-4ec1-8954-3ba1b79a94b4 admin admin] PATH_INFO: `/v3/projects/894f3fdd205b4f77b00b66a7aac48941/users/0483ea16f576469fb462ef11006badb2/roles/a0db5bd603fc4096917bbcd8c6932bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:55.177687 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f9a99-801b-4ec1-8954-3ba1b79a94b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=894f3fdd205b4f77b00b66a7aac48941, user_id=0483ea16f576469fb462ef11006badb2, role_id=a0db5bd603fc4096917bbcd8c6932bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:55.180873 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f9a99-801b-4ec1-8954-3ba1b79a94b4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:55.194821 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1484/2944] 10.222.0.86 () {60 vars in 1483 bytes} [Mon Aug 29 19:22:55 2022] PUT /identity/v3/projects/894f3fdd205b4f77b00b66a7aac48941/users/0483ea16f576469fb462ef11006badb2/roles/a0db5bd603fc4096917bbcd8c6932bc0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:22:55.204975 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385b13-c4aa-4587-9362-cb83b795b1e7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:55.205478 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385b13-c4aa-4587-9362-cb83b795b1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:55.205603 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385b13-c4aa-4587-9362-cb83b795b1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:55.233344 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1d385b13-c4aa-4587-9362-cb83b795b1e7 None None] MFA Rules not processed for user `0483ea16f576469fb462ef11006badb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:22:55.234557 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1d385b13-c4aa-4587-9362-cb83b795b1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:55.298215 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1461/2945] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:22:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:22:55.310052 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f9a99-801b-4ec1-8954-3ba1b79a94b4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:22:55.311773 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7e9f9a99-801b-4ec1-8954-3ba1b79a94b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:22:55.368189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f9a99-801b-4ec1-8954-3ba1b79a94b4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:22:55.371753 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3acd4-a105-4548-a00d-29c3052fcc6c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af69888b04c45b09e34d9ff7a0b564f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:22:55.372345 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3acd4-a105-4548-a00d-29c3052fcc6c service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:22:55.372620 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3acd4-a105-4548-a00d-29c3052fcc6c service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:22:55.372889 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3acd4-a105-4548-a00d-29c3052fcc6c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:22:55.380881 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3acd4-a105-4548-a00d-29c3052fcc6c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:22:55.382777 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3acd4-a105-4548-a00d-29c3052fcc6c service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:22:55.391386 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1485/2946] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:22:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:05.295287 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=40,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:05.387448 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:34.691733 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385b13-c4aa-4587-9362-cb83b795b1e7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:34.753793 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385b13-c4aa-4587-9362-cb83b795b1e7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:34.759053 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:34.759677 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:34.759956 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:34.760228 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] PATH_INFO: `/v3/users/0483ea16f576469fb462ef11006badb2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:34.761336 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0483ea16f576469fb462ef11006badb2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:34.763282 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:34.808944 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0483ea16f576469fb462ef11006badb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:34.821069 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0483ea16f576469fb462ef11006badb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:34.821721 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0483ea16f576469fb462ef11006badb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:34.822317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0483ea16f576469fb462ef11006badb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:34.829394 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0483ea16f576469fb462ef11006badb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:34.842120 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1462/2947] 10.222.0.86 () {58 vars in 1224 bytes} [Mon Aug 29 19:23:34 2022] DELETE /identity/v3/users/0483ea16f576469fb462ef11006badb2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:34.846557 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8562ad-58a4-4f4b-bcf0-452367eb5895 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:34.847082 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8562ad-58a4-4f4b-bcf0-452367eb5895 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:34.847189 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8562ad-58a4-4f4b-bcf0-452367eb5895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:34.866139 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0b8562ad-58a4-4f4b-bcf0-452367eb5895 None None] MFA Rules not processed for user `9c8f433f2f1f4923859810dc0019845c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:23:34.867505 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0b8562ad-58a4-4f4b-bcf0-452367eb5895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:34.925536 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1486/2948] 10.222.0.86 () {58 vars in 958 bytes} [Mon Aug 29 19:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:23:34.935476 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:34.936865 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:35.013229 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459df34-9c5c-469a-9b77-d093a0fe48bf admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:35.017695 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccf754-970c-42f7-b221-df7db286b7a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec64944b34d4ef1bd666bb1b7e216da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:35.018561 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccf754-970c-42f7-b221-df7db286b7a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:35.019091 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccf754-970c-42f7-b221-df7db286b7a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:35.019416 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccf754-970c-42f7-b221-df7db286b7a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:35.029359 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccf754-970c-42f7-b221-df7db286b7a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:35.030773 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccf754-970c-42f7-b221-df7db286b7a5 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:35.040622 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1463/2949] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:23:35.304857 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8562ad-58a4-4f4b-bcf0-452367eb5895 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:23:35.306412 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0b8562ad-58a4-4f4b-bcf0-452367eb5895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:23:35.318978 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8562ad-58a4-4f4b-bcf0-452367eb5895 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:23:35.325419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c8f433f2f1f4923859810dc0019845c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e9f996197414d24b783bd100f720103', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:23:35.325419 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:23:35.325774 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:23:35.325907 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] PATH_INFO: `/v3/projects/894f3fdd205b4f77b00b66a7aac48941` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:23:35.326987 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=894f3fdd205b4f77b00b66a7aac48941)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:23:35.329194 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:23:35.376114 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894f3fdd205b4f77b00b66a7aac48941', 'request_id': 'req-53febdcb-e130-44a8-ba88-fe7fd0107814'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:35.384010 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894f3fdd205b4f77b00b66a7aac48941', 'request_id': 'req-53febdcb-e130-44a8-ba88-fe7fd0107814'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:35.390489 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894f3fdd205b4f77b00b66a7aac48941', 'request_id': 'req-53febdcb-e130-44a8-ba88-fe7fd0107814'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:35.401186 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] The token cache is being invalidate because project 894f3fdd205b4f77b00b66a7aac48941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:23:35.401474 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:35.402095 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-53febdcb-e130-44a8-ba88-fe7fd0107814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:23:35.403583 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1487/2950] 10.222.0.86 () {58 vars in 1233 bytes} [Mon Aug 29 19:23:35 2022] DELETE /identity/v3/projects/894f3fdd205b4f77b00b66a7aac48941 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:23:45.035972 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:23:45.398211 dv-cinder-850830-2-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:25:50.888276 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccf754-970c-42f7-b221-df7db286b7a5 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:25:50.889882 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-30ccf754-970c-42f7-b221-df7db286b7a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:25:50.958206 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccf754-970c-42f7-b221-df7db286b7a5 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:25:51.016622 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62053c3d-f633-4fd1-8e8c-aedf3d329f18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27f771f20b734b679d5553d29a8a3884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0446a852a61f4d2997f9211208ae8756', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:25:51.017235 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62053c3d-f633-4fd1-8e8c-aedf3d329f18 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:25:51.017531 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62053c3d-f633-4fd1-8e8c-aedf3d329f18 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:25:51.017797 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62053c3d-f633-4fd1-8e8c-aedf3d329f18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:25:51.019536 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-62053c3d-f633-4fd1-8e8c-aedf3d329f18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:25:51.030710 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62053c3d-f633-4fd1-8e8c-aedf3d329f18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:25:51.032317 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62053c3d-f633-4fd1-8e8c-aedf3d329f18 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:25:51.042732 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1464/2951] 10.222.0.86 () {60 vars in 1365 bytes} [Mon Aug 29 19:25:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:26:01.037916 dv-cinder-850830-2-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=58 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}